Modcraft - The community dedicated to quality WoW modding!
Legion- / WoD- / Mop- / Cata-Modding => Tools => Topic started by: barncastle on August 14, 2016, 07:24:03 pm
-
A project I started initially to learn how DBC files worked that eventually turned into this and thought it was worth a share.
This editor has full support for reading and saving all release versions of DBC, DB2, WDB and ADB. This does include support for both Legion DB2 and ADB files and works with all variants (header flags) of these.
Like the other editors I've used a definition based system whereby definitions tell the editor how to interpret each file's columns - this is a lot more reliable than guessing column types but does mean the definitions must be maintained. So far, I've mapped almost all expansions with MoP being ~50% complete, WoD being ~95% complete and everything else being 99%+ (excluding column names).
Link to the source can be found here (https://github.com/barncastle/WDBXEditor).
Features:
- Full support of release versions of DBC, DB2, WDB and ADB (WCH3 and WCH4 are not supported as I deem them depreciated)
- Opening and having open multiple files regardless of type and build
- Open DBC/DB2 files from both MPQ archives and CASC directories
- Save single (to file) and save all (to folder)
- Standard CRUD operations as well as go to, copy row, paste row, undo and redo
- Hide, show and sort columns
- A relatively powerful column filter system (similar to boolean search)
- Displaying and editing columns in hex (numeric columns only)
- Exporting to a SQL database, SQL file, CSV file and MPQ archives
- Importing from a SQL database and a CSV file
- An Excel style Find and Replace
- Shortcuts for common tasks using common shortcut key combinations
- A help file to try and cover off some of the pitfalls and caveats of the program (needs some work)
Tools:
- Definition editor for maintaining the definitions
- WotLK Item Import to remove the dreaded red question mark from custom items
- WDB5 Parser which is an attempt to automatically parse the structure of WDB5 files
Things to Note:
- You need .Net 4.6.1 installed (download (https://www.microsoft.com/en-us/download/details.aspx?id=49982))
- Importing gives you the option to import; new rows, changed and new rows and to override all data
- Exporting to MPQ allows you to append to an existing archive or to create a new one
- Currently any SQL import must have identical columns to the file's definition
- Legion ADB files MUST have the DB2 counterpart open before as required information is stored in the DB2 file. The program will prioritise DB2 if DB2 and ADB are opened at the same time
- The WDB5 Parser works surprisingly well thanks to the new field structure data however it does trip up on inline strings so definitions may need to be validated manually
- Undo, redo and copy data history are lost when changing the current file
- Everything is stored in memory so if your PC is ancient the program will crash attempting to read hundreds of files at once!
Credits go to Ladislav Zezula for the awesome StormLib and thanks to all those that contribute to the WoWDev wiki. I've also patched the definitions together for various sources across the internet, there are too many to name, but thanks to all.
Change Log:
V 1.0.1
The following things have been changed/added:
- Added a FileSystemWatcher so that definitions are reloaded as soon as any definition file is saved meaning the application no longer needs to be restarted
- The program now functions correctly if set as the default program for files (thanks to Skarn for this suggestion)
- Find and Replace now uses a lookup table so is much faster
- Specific errors have been refined and made more human
- The program now no longer stops importing if a string is not found in the string table and instead writes "String not found" in the cell's value. This is an attempt to combat some incorrectly modified DB2 files I've come across
- Definitions have been updated for WoD 6.2.4 which is at ~98% now
- A fair amount of optimisation and tinkering
After some advice from Marlamin, Schlumpf and Amaroth I'm going to put the source on git in the coming week so that others can contribute and make this program better!
V 1.0.2
The following things have been changed/added:
- WCH7 (the new Legion ADB format) is now supported for both reading and writing
- A range of bugs and issues have been resolved
- Have added some options to the CSV import to fix erroneous source data:
- Option 1 (default): Increment Ids so none are duplicated
- Option 2: Take the newest record of the duplicated Ids
- The DataGridView has been overhauled meaning it is slightly faster, more memory efficient and now has a cache and state system implemented
- Undo/Redo has been rebuilt and now triggers on all events properly
- Note: Undo/Redo history is still lost on changing file
- Deleting 25+ rows at once is a permanent action (cannot be undone), due to BindingSource + DataGridView performance limitations, and are prompted to complete this action
- New rows now have default values based on their value type and value types are now enforced
- Added a new option to the column filter to hide all empty columns (the eye button)
- Added a clear row right click option which sets the row to default values
- Added the ability to insert a row with a specific Id provided it doesn't exist (Edit Menu/Ctrl + I)
- Added a new line shortcut to avoid having to scroll to the bottom of the grid (Edit Menu/Ctrl + N)
- Have updated the Legion definitions after some issues with WCH7 structures
A big thanks to UncleChristiof and Skarn for ideas, bugs and usability issues!
V 1.0.3
The following things have been changed/added:
The following things have been changed/added:
- WCH8 (the new Legion ADB format) is now supported for both reading and writing
- Save As (F12) has been added meaning Save doesn't prompt for file location
- SQL exprt now works with all "sql_mode"s
- SQL import now automatically fixes NULL values
- Datagrid context menu has been added to all cells as well row headers
- Supports opening multiple selected files at once
- Supports drag and drop files to open (based on Windows permissions)
- Added JSON as an export type
- Added version check based on Github releases
The application now runs as a single instance when launched in GUI mode. This means any files opened thereafter are sent to the running instance instead of opening the application multiple times. In certain situations this is not ideal, so a button has been added to the Load Definition screen which will open a new instance of the application for all files waiting to be loaded.
A range of command line arguments have been added (this is still early stages). This is the beginning of providing an interface so tasks can be automated/invoked via batch scripts and external programs.
Commands currently include (see the help file for more information):
- Console Mode which opens the software in a console session
- Extract which will extract all DBC/DB2 files from a MPQ archive or CASC directory based on a filter string
- Export which exports a specific file to CSV, JSON or SQL
- SQL dump which dumps a file's data directly into a MySQL database table
Thanks to Luzifix for helping with development
(http://www.model-changing.net/uploads/monthly_2016_08/DataGrid.png.0ff1a631ca1d75922a013a9040cb4e89.png)
X-Post to other site: http://www.model-changing.net/files/file/82-wdbx-editor/
-
Great tool, going to test and see differences from the old classic MyDBC. Keep it up.
-
I hope you consider open sourcing this. Too many amazing tools like this one have been abandoned and eventually lost to format changes.
-
I hope you consider open sourcing this. Too many amazing tools like this one have been abandoned and eventually lost to format changes.
Thanks for the compliment! My long term goal is to release the source although this will be after I've finished tinkering and provided it turns out to be a worthwhile tool (haven't had much too much feedback yet) as it needs a bit of refactoring.
That being said; if you do use it and notice any issues, sticking points or missing features let me know!
-
Thanks for the compliment! My long term goal is to release the source although this will be after I've finished tinkering and provided it turns out to be a worthwhile tool (haven't had much too much feedback yet) as it needs a bit of refactoring.
Just on a side note: There rarely, if ever, was anything coming out of refactoring and making code nice before releasing it. In my experience, pushing earlier is better. Even if people don't fork it or anything, there might still be those reading the code, giving helpful comments and such.
-
^^^ this.
-
Really good job
-
F:GamesWoW DC DevelopmentDataPatch-7.MPQDBFilesClientCreatureDisplayInfo.dbc : Failed to enable constraints. One or more rows contain values violating non-null, unique, or foreign-key constraints.
F:GamesWoW DC DevelopmentDataPatch-7.MPQDBFilesClientMap.dbc : Failed to enable constraints. One or more rows contain values violating non-null, unique, or foreign-key constraints.
-
F:GamesWoW DC DevelopmentDataPatch-7.MPQDBFilesClientCreatureDisplayInfo.dbc : Failed to enable constraints. One or more rows contain values violating non-null, unique, or foreign-key constraints.
F:GamesWoW DC DevelopmentDataPatch-7.MPQDBFilesClientMap.dbc : Failed to enable constraints. One or more rows contain values violating non-null, unique, or foreign-key constraints.
Either you have duplicate Ids in your DBC files or the definition has the Id field in the wrong position. Without more info I can't pinpoint this exactly however I'd suggest starting with the definitions; since these are DBC files the Id column should always be the first field.
-
just little add
i hope you didnt used unsigned variables for columns its rare but i rofled hard when i find out various DBC editors faling only because Spell.dbc is using negative values too :D ( for example energy type -2 is your HPs if i remember correctly )
-
Known Database Issue
SQL Error (1364): Field "M_lang_enGB" Doesn't have a default value
Its like that with all with no Default value
-
Known Database Issue
SQL Error (1364): Field "M_lang_enGB" Doesn't have a default value
Its like that with all with no Default value
Thanks for reporting this, it sounds like you've got your MySQL server in STRICT which enforces these kinds of policies. I'll update the program when I get a chance to default strings to a non-null character.
-
Known Database Issue
SQL Error (1364): Field "M_lang_enGB" Doesn't have a default value
Its like that with all with no Default value
Thanks for reporting this, it sounds like you've got your MySQL server in STRICT which enforces these kinds of policies. I'll update the program when I get a chance to default strings to a non-null character.
How would i disable the strict for Non-Null Characters in mysql
-
How would i disable the strict for Non-Null Characters in mysql
If you run select @@sql_mode; it will tell you what mode your engine is running in. Apparently STRICT_TRANS_TABLES is the default now so I'd suggest just updating any text columns to have a default of an empty string literal.
I don't know how many fields you need to alter but the below will generate the SQL needed to bulk update them all for you, obviously replace <table_name> with your table's name.
SELECT GROUP_CONCAT(CONCAT('ALTER TABLE ','<table_name>',' ALTER COLUMN `',COLUMN_NAME,'` SET DEFAULT '''';') SEPARATOR ' ') FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name = '<table_name>' AND DATA_TYPE = 'text';
I've pushed a fix for this to the git repo, I'd be grateful if you had a chance to test it works.
-
is there a compiled update...my visual studieo is to old for it i think.
-
is there a compiled update...my visual studieo is to old for it i think.
Yeah sorry I need to sort my git management out! I've created a release on the master branch https://github.com/barncastle/WDBXEditor/releases/tag/1.0.2
-
is there a compiled update...my visual studieo is to old for it i think.
Yeah sorry I need to sort my git management out! I've created a release on the master branch https://github.com/barncastle/WDBXEditor/releases/tag/1.0.2
Known issue with Certain DBC
CreatureModeldata.dbc
GameObjectDisplayInfo.dbc
the is removed upon import or export..
Example:
Default:
WorldCreatureCreatureNameCreaturename.m2
Exported/Imported:
WorldCreatureCreatureNameCreaturename.m2
Gameobjects have the same issue.
-
I get "An error occured exporting to SQL"
Default Value issue again
-
Nupper, latest version has gone up. I've added a lot of stuff to the SQL Import/Export to try and combat your issues. I've even changed it to temporarily switch the sql mode. Let me know how it goes
-
When you open a dbc with "n" in it, it turns it into the new line character, but does not export it back to "n"
-
Would there be any way to increase the difinition size, as my main work is in db2 files used for a large cata realm with alot of stuff? :)
-
For anyone else who couldn't get this to build, make sure you build it in Visual Studio 2015.