Understanding the DBC Format

[iindigo]

As practice for creating WoW modding tools, I'm trying to write a little DBC Editor, since it seemed like a simple task compared to reading and editing other file types.

So far I can read the file type, number of entries, and number of fields correctly (was much easier than I expected). How do I read each individual entry? The type, entry count, and field count are as simple as grabbing values at certain addresses (0-4, 4-8, and 8-12 to be exact), but how do you deal with strings at varying addresses?

I'm guessing I need to somehow predict the length of each string and then iterate through the data, keeping track of my position?

[schlumpf]

Nope. The strings in the string block are zero terminated. The entries contain an offset.

Code: [Select]

struct foo_dbc_entry
{
  uint32_t id;
  uint32_t some_string;  
};

char* string_block;
foo_dbc_entry* entries;

for (size_t i (0); i < num_entries; ++i)
{
  printf ( "%i: %i, '%s'n"
         , entries[i].id
         , string_block + entries[i].some_string
         );
}

[iindigo]

Thanks for the clarification!

But ouch, reading this makes my head hurt, haha. I'm definitely not used to reading plain C or C++.

So let me get this straight. In this snippet, you are:

1. Defining a DBC entry object to store the id and string data in
2. Defining variables for the string data block and processed entries
3. Looping through the data, pulling strings and printing them until the num_entries int equals the number of zero-terminations

Did I miss anything?

Sorry for bothering you with this elementary-level stuff  :oops: and thanks again.

[schlumpf]

• An entry e={field_i} has field_k of type string.
• DBC has string block B={b_i} and entries E={e_i}.
• Then, an entry has -- for field_k -- the string value [b_{field_k}, b_{field_k + n}] where n >= 0 and b_{field_k + n} = 0 and b_{field_k + l} != 0 for all -1 <= l < n.
• n might be 0 for a zero length string. A string contains a zero terminator in this definition.

[iindigo]

This might take me a while to get my head around. If I were using C++ I could just copy and paste (which is actually bad) but I'm trying to do this in Objective-C which is forcing me to think about it and understand it (obviously good). It's something I'm going to have to do some research on.

Here's the code I have thus far:

Code: [Select]

- (NSDictionary *)readDBCFromData:(NSData *)data
{
    NSData *typeData = [data subdataWithRange:NSMakeRange(0, 4)];
    NSString *typeString = [NSString stringWithUTF8String:[typeData bytes]];
   
    NSDictionary *dataDictionary;
   
    if ([typeString isEqualToString:@"WDBC"]) {
       
          NSData    *recordCountData = [data subdataWithRange:NSMakeRange(4, 8)];
        NSNumber    *recordCount     = [NSNumber numberWithInt:*(int*)([recordCountData bytes])];
          NSData    *fieldCountData  = [data subdataWithRange:NSMakeRange(8, 12)];
        NSNumber    *fieldCount      = [NSNumber numberWithInt:*(int*)([fieldCountData bytes])];
       
        dataDictionary = @{ @"recordCount":recordCount, @"fieldCount":fieldCount };
    }

    return dataDictionary;
}
Which, after some UI code, looks like this:

[schlumpf]

I updated the wiki article to be more clear. See http://wowdev.wiki/index.php?title=Category:DBC.

You might want to change

Code: [Select]

NSData    *recordCountData = [data subdataWithRange:NSMakeRange(4, 8)];
NSNumber    *recordCount     = [NSNumber numberWithInt:*(int*)([recordCountData bytes])];to

Code: [Select]

NSNumber* recordCount = [NSNumber numberWithInt:*(int*)([data bytes] + 4)];
In your case, you might want to use the second (offset based) variant in Objective-C. NSString's stringWithUTF8String: should be used.

Code: [Select]

NSString* name = [NSString stringWithUTF8String:([data bytes] + calculated_offset)];

[iindigo]

The update is quite helpful, thanks.

This confuses me, though:

Code: [Select]

NSString* name = [NSString stringWithUTF8String:([data bytes] + calculated_offset)];
What is this a parallel to in the C++? Also, should I be using structs in Obj-C? I know it supports them but I've had no need for them up until this point.

[schlumpf]

This confuses me, though:

Code: [Select]

NSString* name = [NSString stringWithUTF8String:([data bytes] + calculated_offset)];

Code: [Select]

// const char* file;
// const char* records;
uint32_t record_count = *(uint32_t*) (file + 1 * sizeof (uint32_t));
uint32_t record_size = *(uint32_t*) (file + 3 * sizeof (uint32_t));
uint32_t string_block_offset = 5 * sizeof (uint32_t) + record_size * record_count;
uint32_t string_offset = *(uint32_t*)(records + i * record_size + sizeof (uint32_t) /* name */);
uint32_t calculated_offset = string_block_offset + string_offset;
printf ("record %u: %u, %sn", *(uint32_t*)(records + i * record_size /* id */), file + calculated_offset);

Also, should I be using structs in Obj-C? I know it supports them but I've had no need for them up until this point.

That would be personal preference. It might be easier using them, as you can just take [data bytes] once, cast it to dbc_file<T> and you're done. This does assume you know T though. Else, there isn't that much benefit.

You may want to take

Code: [Select]

NSData* string_block = [data subdataWithRange:NSMakeRange(string_block_offset, [data count])];though. Then, you will just have to take

Code: [Select]

[NSString stringWithUTF8String:([string_block bytes] + string_offset)];

[iindigo]

Recently I've had the time to pick this up and start tinkering with it again and the situation hasn't changed much from what it was before. I still feel like an idiot swinging around in the dark because of bits and pieces I don't understand.

Code: [Select]

// const char* file;
// const char* records;
uint32_t record_count = *(uint32_t*) (file + 1 * sizeof (uint32_t));
uint32_t record_size = *(uint32_t*) (file + 3 * sizeof (uint32_t));
uint32_t string_block_offset = 5 * sizeof (uint32_t) + record_size * record_count;
uint32_t string_offset = *(uint32_t*)(records + i * record_size + sizeof (uint32_t) /* name */);
uint32_t calculated_offset = string_block_offset + string_offset;
printf ("record %u: %u, %sn", *(uint32_t*)(records + i * record_size /* id */), file + calculated_offset);

In this, what is records? Is it the same as what's listed in the wiki entry, i.e.

Code: [Select]

const char* records = file + 5 * sizeof (uint32_t) /* header */;? What is its role in calculating the string offset?

Also, do values of 264, 35924, and 43533 sound right for record size, string block offset, and length of the file in bytes? The DBC file I'm testing with is attached.

[schlumpf]

Records is a pointer to the individual rows of the database, as described in the specific table documentation. Records start behind the header, so yes, it is file + 5*sizeof uint32
The string block starts behind the records, thus is at records + record_count * record_size.
Numbers seem fine, if there are about 66 columns and 136 rows.

[iindigo]

Thanks for the help, I'm finally getting somewhere!



Would I be correct in guessing that DBC reading has to be done in a file-specific way (for example, the implementation for reading Map.dbc is different from that for reading AreaTable.dbc)? I don't see any other way to get around the arbitrary mix of ints and strings.

EDIT: nevermind, I'm dumb, it's all C strings
currently playing around trying to figure out how to pull in the other fields

[schlumpf]

It is right, that you need table specific behavior. WoW does include meta descriptions of the tables, but it is far from enough to write a generic editor from it. The wiki exists for a reason: Tables are different and not trivially reverse engineered.

[iindigo]

Guess I'll have to include some sort of profile functionality then. Not hard, just time consuming.

Thanks again for the assistance... I have one more question after which I'll try to leave you alone. How does one move back and forth between columns? That's the last thing that I really fail to understand.

[Steff]

You should use definition files for every DBC. Including:

Fild
Name
Type (Int, Float, String, LocString, Flags)
Ref table,RefFild ID, RefFildName

Thats what I did in my Genesis editor.

I also thing about a flag file to define them.

[schlumpf]

How does one move back and forth between columns? That's the last thing that I really fail to understand.

I fail to understand the question, though.

Also, as you probably write an editor for a fixed version of WoW, you may even hard-code the table definitions.