Modcraft - The community dedicated to quality WoW modding!

Wrath of the Lich King Modding => Miscellaneous => Topic started by: Rangorn on April 13, 2016, 11:52:02 am

Title: [Legion] WDB5 format
Post by: Rangorn on April 13, 2016, 11:52:02 am
[deleted]
Title: Re: [Leagion]  WDB4 format
Post by: Steff on April 13, 2016, 12:58:01 pm
Please. IF you discover stuff. Create and dev wiki account and document it.
If we not build on each others discovery we will never get to a good point.
Title: Re: [Leagion]  WDB4 format
Post by: schlumpf on April 13, 2016, 02:02:14 pm
Note that the layout of tables can (has and will again) change depending on the data contained in a client patch. Columns are compressed to fit the maximum value in them. If the maximum value changes, the column layout will change. If you reverse the layout for a table for a given build, please add it to the wiki, e.g. at wowdev.wiki/DB/ChrClasses in your case.
Title: Re: [Leagion]  WDB4 format
Post by: Rangorn on April 13, 2016, 02:05:03 pm
[deleted]
Title: Re: [Leagion]  WDB4 format
Post by: Rangorn on April 13, 2016, 02:09:19 pm
[deleted]
Title: Re: [Legion] WDB4 format
Post by: Rangorn on April 13, 2016, 05:39:22 pm
I finally switch to another DB2, CurrencyTypes.db2, same columns but with different size / order
That was easy for this one, i'm 100% sure that's the good structure.

https://wowdev.wiki/DB/CurrencyTypes (https://wowdev.wiki/DB/CurrencyTypes" onclick="window.open(this.href);return false;)
Title: Re: [Legion] WDB4 format
Post by: schlumpf on April 13, 2016, 06:07:27 pm
In general, the order of columns should be the same, but with stable-sorting by type size.
Title: Re: [Legion] WDB4 format
Post by: Rangorn on April 14, 2016, 06:33:33 pm
New Legion Build...

Now it's WDB5

*dies internally*
Title: Re: [Legion] WDB5 format
Post by: Rangorn on April 18, 2016, 02:18:05 am
[deleted]
Title: Re: [Legion] WDB5 format
Post by: schlumpf on April 18, 2016, 10:31:42 am
Uint40 sounds wrong
Title: Re: [Legion] WDB5 format
Post by: Rangorn on April 18, 2016, 03:55:17 pm
[deleted]
Title: Re: [Legion] WDB5 format
Post by: schlumpf on April 18, 2016, 08:37:31 pm
Weird shit, but okay, that's blizzard.
Title: Re: [Legion] WDB5 format
Post by: Simca on April 18, 2016, 09:04:28 pm
It is uint16; uint40 is an illusion created by record alignment.

When Blizzard writes records to files, the end of the record is aligned to the largest field size used.

In this file in question, the ID field starts at 0x1B into the record and lasts for 2 bytes according to the field structure block ((32 - 0x10) / 8 = 2). This puts the end of the field at 0x1D. However, the record size field in the header says the record is 0x20. The difference is because of the alignment I just mentioned. The largest field size used in the record is 4 bytes. The nearest values divisible by 4 bytes are 0x1C (which the ID field goes past) and 0x20, so the record is padded by 3 bytes to make it properly align at 0x20.

So yes, you can read it with that structure, but it would be better to read it as uint16 and then skip to the end of the record.
Title: Re: [Legion] WDB5 format
Post by: schlumpf on April 18, 2016, 09:41:46 pm
Also, get onto irc, as you seem to be one of the intrersting people doing stuff (^ー^)ノ
Title: Re: [Legion] WDB5 format
Post by: Rangorn on April 18, 2016, 11:54:55 pm
Quote from: "Simca"
It is uint16; uint40 is an illusion created by record alignment.

When Blizzard writes records to files, the end of the record is aligned to the largest field size used.

In this file in question, the ID field starts at 0x1B into the record and lasts for 2 bytes according to the field structure block ((32 - 0x10) / 8 = 2). This puts the end of the field at 0x1D. However, the record size field in the header says the record is 0x20. The difference is because of the alignment I just mentioned. The largest field size used in the record is 4 bytes. The nearest values divisible by 4 bytes are 0x1C (which the ID field goes past) and 0x20, so the record is padded by 3 bytes to make it properly align at 0x20.

So yes, you can read it with that structure, but it would be better to read it as uint16 and then skip to the end of the record.

Ok i get it, thanks for your explanation :)
I think this could be helpful to write a fake-column with the amount of padding for each DB
Title: Re: [Legion] WDB5 format
Post by: schlumpf on April 19, 2016, 01:15:17 am
According to the c and c++ standards the padding is implicit by just writing the struct. I think that if we want to mentio it, it should be via ((_aligned_(32))) pragma or alike. It isn't table specific but just a language feature. I realize it can lead to confusion.
Title: Re: [Legion] WDB5 format
Post by: Rangorn on April 19, 2016, 08:40:20 pm
Ok i'm done with WDB5

The furry already made a program for parsing All the files into csv format
https://github.com/Arctium-Emulation/Tools (https://github.com/Arctium-Emulation/Tools" onclick="window.open(this.href);return false;)

I will maybe only contribute to the wiki by referencing structure + column name.
Title: Re: [Legion] WDB5 format
Post by: Rangorn on April 28, 2016, 04:37:15 pm
[deleted]
Title: Re: [Legion] WDB5 format
Post by: schlumpf on April 28, 2016, 07:19:14 pm
That change happened between cata and wod, to be exact ;)
Title: Re: [Legion] WDB5 format
Post by: Uthil on April 28, 2016, 09:15:31 pm
Build 21531 has WDB5, im looking for an editor/viewer or something similar. All my search end up to find up too wdb2, thats already outdated. It will possible to read WDB5 from now on ?? Since blizz tries to complex things more and more as heading to close legion.
Title: Re: [Legion] WDB5 format
Post by: Rangorn on April 29, 2016, 01:04:46 am
[deleted]
Title: Re: [Legion] WDB5 format
Post by: schlumpf on April 29, 2016, 08:10:42 am
That comes from how they count "columns". It was changed from four fields to an array of four before/in wod. Counting an array as just one column changed in wdb6. So yes, it now shows that it is an array, but the struct used was using an array before
Title: Re: [Legion] WDB5 format
Post by: Steff on May 02, 2016, 11:48:30 am
They started earlierwith this grouping. CharRace for example in wod is also in this form.
Title: Re: [Legion] WDB5 format
Post by: Rangorn on May 03, 2016, 07:30:54 pm
Yeah, but i'm talking about the header
Title: Re: [Legion] WDB5 format
Post by: Rangorn on May 04, 2016, 06:34:18 pm
[deleted]
Title: Re: [Legion] WDB5 format
Post by: Simca on May 04, 2016, 09:01:19 pm
Yeah, basically old school DBC column guessing strategy.

I use a priority-based system, like:
1. If every value in the column is 0, it's an integer.
2. If every value in the column exists in our string position map, it's a string.
3. If every value in the column where it is not 0 has the exponent bits for floats set to a number greater than ~120, it's a float.
4. Else, it's an integer.

A couple notes for WDB5: string and float fields still -must- be 4 bytes. So if the field is less than 4 bytes, it is automatically an integer.

Also, there are going to be a lot of problems with signed and unsigned integers if you autodetect like this. Foreign keys are generally unsigned and it sometimes makes a big difference. Other values are generally signed, and it sometimes makes a big difference. You might be able to write some form of autodetection for signed vs unsigned (or take it from somewhere else) but it is going to be annoyingly complicated.

Lastly, I recommend against a dedicated flags column type. At the end of the day, displaying the numbers as 0x(some hex value) doesn't actually make flags significantly more clear or useful (imo), so it's best to just parse flags you want later, after converting to CSV or some other form.
Title: Re: [Legion] WDB5 format
Post by: Rangorn on May 05, 2016, 02:22:06 am
[deleted]
Title: Re: [Legion] WDB5 format
Post by: Steff on May 05, 2016, 08:39:52 pm
I think it is great if your tool supports gussing but you should also implement the posibillity to rework your gusses.
Just save the structure in a xml.csv whatever format and let the uset change the names of columns and the types.
Title: Re: [Legion] WDB5 format
Post by: Rangorn on May 12, 2016, 11:26:39 am
[deleted]
Title: Re: [Legion] WDB5 format
Post by: schlumpf on May 12, 2016, 11:49:39 am
texturefiledata or something
Title: Re: [Legion] WDB5 format
Post by: Rangorn on May 12, 2016, 02:41:10 pm
[deleted]