Modcraft - The community dedicated to quality WoW modding!
Projects => Software Development => Topic started by: iindigo on October 03, 2012, 01:48:17 pm
-
As practice for creating WoW modding tools, I'm trying to write a little DBC Editor, since it seemed like a simple task compared to reading and editing other file types.
So far I can read the file type, number of entries, and number of fields correctly (was much easier than I expected). How do I read each individual entry? The type, entry count, and field count are as simple as grabbing values at certain addresses (0-4, 4-8, and 8-12 to be exact), but how do you deal with strings at varying addresses?
I'm guessing I need to somehow predict the length of each string and then iterate through the data, keeping track of my position?
-
Nope. The strings in the string block are zero terminated. The entries contain an offset.
struct foo_dbc_entry
{
uint32_t id;
uint32_t some_string;
};
char* string_block;
foo_dbc_entry* entries;
for (size_t i (0); i < num_entries; ++i)
{
printf ( "%i: %i, '%s'n"
, entries[i].id
, string_block + entries[i].some_string
);
}
-
Thanks for the clarification!
But ouch, reading this makes my head hurt, haha. I'm definitely not used to reading plain C or C++.
So let me get this straight. In this snippet, you are:
1. Defining a DBC entry object to store the id and string data in
2. Defining variables for the string data block and processed entries
3. Looping through the data, pulling strings and printing them until the num_entries int equals the number of zero-terminations
Did I miss anything?
Sorry for bothering you with this elementary-level stuff :oops: and thanks again.
-
- An entry e={field_i} has field_k of type string.
- DBC has string block B={b_i} and entries E={e_i}.
- Then, an entry has -- for field_k -- the string value [b_{field_k}, b_{field_k + n}] where n >= 0 and b_{field_k + n} = 0 and b_{field_k + l} != 0 for all -1 <= l < n.
- n might be 0 for a zero length string. A string contains a zero terminator in this definition.
-
This might take me a while to get my head around. If I were using C++ I could just copy and paste (which is actually bad) but I'm trying to do this in Objective-C which is forcing me to think about it and understand it (obviously good). It's something I'm going to have to do some research on.
Here's the code I have thus far:
- (NSDictionary *)readDBCFromData:(NSData *)data
{
NSData *typeData = [data subdataWithRange:NSMakeRange(0, 4)];
NSString *typeString = [NSString stringWithUTF8String:[typeData bytes]];
NSDictionary *dataDictionary;
if ([typeString isEqualToString:@"WDBC"]) {
NSData *recordCountData = [data subdataWithRange:NSMakeRange(4, 8)];
NSNumber *recordCount = [NSNumber numberWithInt:*(int*)([recordCountData bytes])];
NSData *fieldCountData = [data subdataWithRange:NSMakeRange(8, 12)];
NSNumber *fieldCount = [NSNumber numberWithInt:*(int*)([fieldCountData bytes])];
dataDictionary = @{ @"recordCount":recordCount, @"fieldCount":fieldCount };
}
return dataDictionary;
}
Which, after some UI code, looks like this:
(http://iindigo3d.com/dbcreader.png)
-
I updated the wiki article to be more clear. See http://pxr.dk/wowdev/wiki/index.php?title=Category:DBC (http://pxr.dk/wowdev/wiki/index.php?title=Category:DBC" onclick="window.open(this.href);return false;).
You might want to change
NSData *recordCountData = [data subdataWithRange:NSMakeRange(4, 8)];
NSNumber *recordCount = [NSNumber numberWithInt:*(int*)([recordCountData bytes])];to
NSNumber* recordCount = [NSNumber numberWithInt:*(int*)([data bytes] + 4)];
In your case, you might want to use the second (offset based) variant in Objective-C. NSString's stringWithUTF8String: should be used.
NSString* name = [NSString stringWithUTF8String:([data bytes] + calculated_offset)];
-
The update is quite helpful, thanks.
This confuses me, though:
NSString* name = [NSString stringWithUTF8String:([data bytes] + calculated_offset)];
What is this a parallel to in the C++? Also, should I be using structs in Obj-C? I know it supports them but I've had no need for them up until this point.
-
This confuses me, though:
NSString* name = [NSString stringWithUTF8String:([data bytes] + calculated_offset)];
// const char* file;
// const char* records;
uint32_t record_count = *(uint32_t*) (file + 1 * sizeof (uint32_t));
uint32_t record_size = *(uint32_t*) (file + 3 * sizeof (uint32_t));
uint32_t string_block_offset = 5 * sizeof (uint32_t) + record_size * record_count;
uint32_t string_offset = *(uint32_t*)(records + i * record_size + sizeof (uint32_t) /* name */);
uint32_t calculated_offset = string_block_offset + string_offset;
printf ("record %u: %u, %sn", *(uint32_t*)(records + i * record_size /* id */), file + calculated_offset);
Also, should I be using structs in Obj-C? I know it supports them but I've had no need for them up until this point.
That would be personal preference. It might be easier using them, as you can just take [data bytes] once, cast it to dbc_file<T> and you're done. This does assume you know T though. Else, there isn't that much benefit.
You may want to take
NSData* string_block = [data subdataWithRange:NSMakeRange(string_block_offset, [data count])];though. Then, you will just have to take
[NSString stringWithUTF8String:([string_block bytes] + string_offset)];
-
Recently I've had the time to pick this up and start tinkering with it again and the situation hasn't changed much from what it was before. I still feel like an idiot swinging around in the dark because of bits and pieces I don't understand.
// const char* file;
// const char* records;
uint32_t record_count = *(uint32_t*) (file + 1 * sizeof (uint32_t));
uint32_t record_size = *(uint32_t*) (file + 3 * sizeof (uint32_t));
uint32_t string_block_offset = 5 * sizeof (uint32_t) + record_size * record_count;
uint32_t string_offset = *(uint32_t*)(records + i * record_size + sizeof (uint32_t) /* name */);
uint32_t calculated_offset = string_block_offset + string_offset;
printf ("record %u: %u, %sn", *(uint32_t*)(records + i * record_size /* id */), file + calculated_offset);
In this, what is records? Is it the same as what's listed in the wiki entry, i.e.
const char* records = file + 5 * sizeof (uint32_t) /* header */;? What is its role in calculating the string offset?
Also, do values of 264, 35924, and 43533 sound right for record size, string block offset, and length of the file in bytes? The DBC file I'm testing with is attached.
-
Records is a pointer to the individual rows of the database, as described in the specific table documentation. Records start behind the header, so yes, it is file + 5*sizeof uint32
The string block starts behind the records, thus is at records + record_count * record_size.
Numbers seem fine, if there are about 66 columns and 136 rows.
-
Thanks for the help, I'm finally getting somewhere!
(http://f.cl.ly/items/3b130H3y0g3B3C40161k/dbcedit.png)
Would I be correct in guessing that DBC reading has to be done in a file-specific way (for example, the implementation for reading Map.dbc is different from that for reading AreaTable.dbc)? I don't see any other way to get around the arbitrary mix of ints and strings.
EDIT: nevermind, I'm dumb, it's all C strings
currently playing around trying to figure out how to pull in the other fields
-
It is right, that you need table specific behavior. WoW does include meta descriptions of the tables, but it is far from enough to write a generic editor from it. The wiki exists for a reason: Tables are different and not trivially reverse engineered.
-
Guess I'll have to include some sort of profile functionality then. Not hard, just time consuming.
Thanks again for the assistance... I have one more question after which I'll try to leave you alone. How does one move back and forth between columns? That's the last thing that I really fail to understand.
-
You should use definition files for every DBC. Including:
Fild
Name
Type (Int, Float, String, LocString, Flags)
Ref table,RefFild ID, RefFildName
Thats what I did in my Genesis editor.
I also thing about a flag file to define them.
-
How does one move back and forth between columns? That's the last thing that I really fail to understand.
I fail to understand the question, though.
Also, as you probably write an editor for a fixed version of WoW, you may even hard-code the table definitions.
-
Perhaps a better way to phrase my question is, "What part of the code changes the returned column?" I've tried figuring this out by tinkering with different aspects but nothing seems to work.
Though this is an editor for 3.3.5 (at least initially) I plan to allow adding, removal, and editing of profiles. The core profiles will be included as defaults that can be reverted to at any point (really easy to do with NSUserDefaults).
Steff, this is what my profile editor looks like so far. What's missing? Going by the list you've provided perhaps this is too simple.
(http://f.cl.ly/items/120A0B1d2344093Q111T/Screen%20Shot%202013-04-08%20at%209.01.37%20AM.png)
EDIT: After more tinkering, I figured it out! You just increment by the length of an int. Very simple.
(http://f.cl.ly/items/0l2Q0Y0A2u1R343J2B3D/Screen%20Shot%202013-04-08%20at%202.49.28%20PM.png)
My next question, then, is what do flags count as? They're not ints or strings as far as I can tell
EDIT 2: Nevermind, it's just an int in hex format
-
I use an open and external format to:
1 Change it fast if new stuff is invented.
2 Other people can help me to create the definitions.
I also use them just to import the dbcs into mysql and edit them then there.
-
I use an open and external format to:
1 Change it fast if new stuff is invented.
2 Other people can help me to create the definitions.
I also use them just to import the dbcs into mysql and edit them then there.
I'm planning to allow importing and exporting of profiles in XML format, does that sound good?
-
I use an open and external format to:
1 Change it fast if new stuff is invented.
2 Other people can help me to create the definitions.
I also use them just to import the dbcs into mysql and edit them then there.
I'm planning to allow importing and exporting of profiles in XML format, does that sound good?
Use .txt or .ini - it is not a special format like XML and does not require overly complicated instructions to form proper tables.
-
I use an open and external format to:
1 Change it fast if new stuff is invented.
2 Other people can help me to create the definitions.
I also use them just to import the dbcs into mysql and edit them then there.
I'm planning to allow importing and exporting of profiles in XML format, does that sound good?
Use .txt or .ini - it is not a special format like XML and does not require overly complicated instructions to form proper tables.
I see your point. I might do multiple formats. Text will actually take me more time to write since there's built-in functions for reading and writing XML.
-
I've been working on this again lately and have been making nice headway. It's nowhere near complete but it's approaching usability. The definitions editor is almost finished as is mapping... there are a few types that aren't mapped and cross-file references aren't working yet so those will be the next targets.
(http://f.cl.ly/items/0R0u2O2z1X2u1N3H073i/Screen%20Shot%202014-01-19%20at%201.15.02%20PM.png)
Click to Enlarge (http://f.cl.ly/items/0R0u2O2z1X2u1N3H073i/Screen%20Shot%202014-01-19%20at%201.15.02%20PM.png)
-
Looks good.
-
When you get to mapping the Spell.DBC file, hit me up.
I have extensive documentation on how that DBC works.
-
If you have more information about something than it is in the wiki at pxr.dk/wowdev/wiki, please add it there! Holding information in private does not help anyone, even if you share it upon private messaging request!
-
I don't have any new information that would help out at WoW Dev Wiki, I least I don't think it could help. Ive been using the TrinityCore Source code and QTSpellWork program to figure out what the Effect columns do for each spell Effect and Aura.
It won't be private much longer, I've already shown Steff what I have. But right now its nothing more than a collection of notes on about 100 different Auras and Effects. I'm not going to release something that isn't useful in it's current state.
-
As you can see in the pm, it does contain information not on the wiki, thus is helpful.