Modcraft - The community dedicated to quality WoW modding!

Wrath of the Lich King Modding => Resources and Tools => Topic started by: spik96 on July 04, 2015, 01:21:43 am

Title: [TOOL] LKBC M2 Converter v0.4
Post by: spik96 on July 04, 2015, 01:21:43 am
** THIS PROJECT IS DEAD AND REPLACE BY JM2CONVERTER WHICH IS MORE POWERFUL, SEE GITHUB **

Hi Modcraft !
Now is the time to release the tool I've been working on for a couple of months.

##What it is :
World of Warcraft M2 model converter from version 08 (_Wrath of the Lich King_) to 07-04 (_The Burning Crusade_).
Written in ANSI C, documented with Doxygen, made with love and tears.

##How to use it :
This is a command line tool. Put the executable file in the same folder than your WotLK model, then run it in a terminal. The output will be <model name>_BC.m2.

###Example (Windows version)
Original model : PandarenStag.m2
Code: [Select]
lkbc_converter.exe PandarenStag.m2==> a new file, PandarenStag_BC.m2, should have appeared in your folder.
Note : if you can't understand this, I recommend you to not bother with this program and learn basic knowledge like how to open a terminal, how to navigate in it...
Drag&Drop in Explorer may also work. Untested.

##Things that should work :
* Creatures, with their animations
* Static objects like trees or crates
* Objects with animations like fountains or waterfalls
* Interface m2 files

##What's yet to be implemented :
* Ribbons
* Particles
Note : I tried to make it so the converter will still produce a model, you'll just not be able to see the effects.

This is still heavy work in progress.
 
##Frequently Asked Questions :
###Can we convert Warlords of Draenor character models ?
A bit buggy but theoritically yes. But :
* There are geosets supported by LK and not by BC. As a result, if you don't delete them in Blender, they will always show. Examples : "Catch-like" belt, Death Knights glowing eyes.
* Remember you don't have to decimate character models meshes if you convert to BC because there a lot of unused sets. Haircuts (those added with the barbershop in LK) for example.
* You will have to resize textures to the size of those of classic models. That's a huge loss in quality, but there is nothing we can do about it. Character texture size is hardcoded in the client. If you don't reduce them, you'll get a WoW Error or, if you're lucky, messy faces.
* Don't forget the DBC Editing. You can't use LK DBCs.

###Can we use it on creatures or only with static objects ?
This converted is aimed at converting creatures, something that had never been done previously by any LKBC conversion tool.
Not all of them are working perfectly, but the code is getting better every day.

###My model is not working !
Check you're not using a model which has unsupported features, like particles or ribbons.
But errors can happen elsewhere too. You can report your problem in the issue section on Github ;) !
Don't forget to add the name of your model and the nature of the problem.
 

##Credits :
Original idea and structures by Stan84.
Thanks to Stan84, PhilipTNG, Schlumpf and all the people who contributed to [wiki:2tqnq0t6]M2[/wiki:2tqnq0t6]&[wiki:2tqnq0t6]M2/WotLK[/wiki:2tqnq0t6] pages.

##Download :
(READY-TO-USE FILES ARE IN THE "BINARIES" SECTION)
https://github.com/Koward/LKBC_Converter/releases (https://github.com/Koward/LKBC_Converter/releases" onclick="window.open(this.href);return false;)
Available for Windows and Linux !
Title: Re: [TOOL] LKBC M2 Converter
Post by: abdalrahman9 on July 04, 2015, 09:39:49 am
Hey there, now i am not that into BC but its cool to see something like this. I just wanted to comment on the following:
Quote from: "spik96"
##Frequently Asked Questions :
###Can we convert Warlords of Draenor character models ?
Not yet. I don't know if we'll ever be able to. It would be the pinnacle of this project.
Can't you just convert it from WOD to WOTLK and then from WOTLK to BC with this converter? Or am i missing something
Title: Re: [TOOL] LKBC M2 Converter
Post by: spik96 on July 04, 2015, 10:05:27 am
Quote from: "abdalrahman9"
Hey there, now i am not that into BC but its cool to see something like this. I just wanted to comment on the following:
Quote from: "spik96"
##Frequently Asked Questions :
###Can we convert Warlords of Draenor character models ?
Not yet. I don't know if we'll ever be able to. It would be the pinnacle of this project.
Can't you just convert it from WOD to WOTLK and then from WOTLK to BC with this converter? Or am i missing something

You're missing the AnimIDs. They are a lot of animations in these new models with IDs&Flags&Whatever meaning nothing to a little BC client from 8 years ago. Like Monk animations ! I don't handle this problem yet.
Note this problem could also occur on non character models. I have ideas, however, but it will take a little more time.
This is really a pre-release so yeah bugs can (and will) happen. I just consider it in a useful-enough state to be shared with the world.

EDIT : I made some tests and the conversion is not possible at the moment for most HD character models, and none is possible with the current public version. If it's possible, I will try to fix it for the next update.
Title: Re: [TOOL] LKBC M2 Converter
Post by: PhilipTNG on July 04, 2015, 06:16:25 pm
Good Job Spik!! Well done man, glad you came out of this ordeal with your sanity intact!!

Quote from: "abdalrahman9"
Can't you just convert it from WOD to WOTLK and then from WOTLK to BC with this converter? Or am i missing something
There's a reason why there isn't a text or video guide on how to convert LK to TBC. And why it took us a crazy long time to figure out how to get stuff to show up ingame in the first place.

The data format is quite different, just because it says M2, doesn't mean it's using the same data structure. Take for instance, in LK and beyond, you have anim files and skin files. in BC, the data for these files were originally a part of the M2 file, meaning these extra files didn't exist in BC and Vanilla.  So they have to be merged back into the M2 file to work in BC, but you also need to set up the offsets to the data, you can't simply append the data to the end of the file, It has to be added in a specific order and you have to set up the callout address in the header so the client knows how to handle them.

In LK and beyond, various data structures use 2 location jump offsets before it gets to the actual data it's using. in BC, it used 1, So if that doesn't make sense, think of it this way, there will be hundreds of things that use 2 callouts, and to make it work in bc, you need to compromise those hundreds of offsets to a single offsets for each section.

For particles for example, for any of the particle options, it will generally give you a jump offset to another location that has another jump offset that points to the data it's supposed to use.  in BC, there's only one jump offset that goes tot he data it's supposed to use, Each particle has 10-12 of these that you must do compromises on, to make it so the first jump ends up at the last area and not the second jump because there's no usable data there.  That's kind of why you can't simply make a tutorial or a guide on how to convert.
Title: Re: [TOOL] LKBC M2 Converter v0.2
Post by: abdalrahman9 on July 04, 2015, 07:54:41 pm
Ow i see, i guess there was more to it than i thought. Thanks for the reply you two!
Title: Re: [TOOL] LKBC M2 Converter
Post by: spik96 on July 04, 2015, 08:26:04 pm
Quote from: "PhilipTNG"
Good Job Spik!! Well done man, glad you came out of this ordeal with your sanity intact!

I don't know if my sanity is really intact to be honest xD !
Converting from multiple timelines (LK) to one timeline (BC) in AnimBlocks was pretty hard. When I saw there were dozens of them for the model, I spent one whole week refactoring. Now I can just use a function (example :    
Code: [Select]
convert_Vec3DAnimBlock(lk_m2.bones[i].trans, lk_m2.animofs[i].trans,
lk_m2.bonesdata[i].trans, &ptr->bones[i].trans,
&ptr->bonesdata[i].trans, animations, nAnimations);

If you or anybody want to develop something that may use this (maybe MDX or M3 models ??), take a look at my Github cause I wish I had something similar to start from x) !

EDIT : version 0.3 released, with way less bugs. Check it on https://github.com/Koward/LKBC_Converter/releases (https://github.com/Koward/LKBC_Converter/releases" onclick="window.open(this.href);return false;) .
Title: Re: [TOOL] LKBC M2 Converter v0.3
Post by: Helkar on July 06, 2015, 11:53:45 am
Nothing is clear ! Make instructions
translation by google translator
Title: Re: [TOOL] LKBC M2 Converter v0.3
Post by: spik96 on July 06, 2015, 12:15:45 pm
Quote from: "Helkar"
Nothing is clear ! Make instructions
translation by google translator

Instructions added. I will add a screen-shot tomorrow.
Title: Re: [TOOL] LKBC M2 Converter v0.3
Post by: PhilipTNG on July 11, 2015, 04:40:21 pm
Quote from: "Helkar"
Nothing is clear ! Make instructions
translation by google translator
lol. what is so hard about drag and drop?
Title: Re: [TOOL] LKBC M2 Converter v0.3
Post by: spik96 on July 11, 2015, 06:55:20 pm
The next update, 0.4, will have some commands like "-t" to show required textures for your model and also "-c" to convert to Classic instead of TBC. More things like TexAnims and Lights will also be converted. And tons of bugfixes of course.
The list will show when you enter no argument and there will also be indications on how to use the program.
Title: Re: [TOOL] LKBC M2 Converter v0.3
Post by: Gurluas on July 11, 2015, 08:25:47 pm
Any chance you could make a program do the opposite too?
I.E TBC to LK, there is no way to do that atm.
Title: Re: [TOOL] LKBC M2 Converter v0.3
Post by: spik96 on July 11, 2015, 08:28:57 pm
Quote from: "Gurluas"
Any chance you could make a program do the opposite too?
I.E TBC to LK, there is no way to do that atm.

That would be pointless work. The models from the TBC client are all in the WotLK client, and the tools for custom models are for the LK M2 format. It's not worth the time to implement it.
Title: Re: [TOOL] LKBC M2 Converter v0.3
Post by: Gurluas on July 15, 2015, 11:38:27 pm
Quote from: "spik96"
Quote from: "Gurluas"
Any chance you could make a program do the opposite too?
I.E TBC to LK, there is no way to do that atm.

That would be pointless work. The models from the TBC client are all in the WotLK client, and the tools for custom models are for the LK M2 format. It's not worth the time to implement it.

The TBC Alpha models are not in Wotlk, nor can they be properly ported due to limitations in the tools (Player models have many skin files).
And some of them have vast differences in animation or even body.

And theres also Wotlk Alpha/Beta stuff, like the Taunka with a boner, and the Neanderthal'ish Vrykul.
Title: Re: [TOOL] LKBC M2 Converter v0.3
Post by: spik96 on July 16, 2015, 10:22:47 am
Quote from: "Gurluas"
The TBC Alpha models are not in Wotlk, nor can they be properly ported due to limitations in the tools (Player models have many skin files).
And some of them have vast differences in animation or even body.

And theres also Wotlk Alpha/Beta stuff, like the Taunka with a boner, and the Neanderthal'ish Vrykul.

TBC Alpha models : open them with MDLVis, save as .mdx, follow Skarn's tutorial to get them in game.
WotLK Alpha : with 010 editor change 08 to 04, you'll then be able to open them with MDLVis too.

This is not an ideal situation. The best thing would be a MDX>M2 converter not completely bugged. I don't think I'm able to do it yet (mostly due to a lack of C++ knowledge not allowing me to use wc3lib), but possibly in the future.
Even if going through Skarn's method can be hard, it's, at least, a solution ATM. There was absolutely no solution for the TBC and Vanilla modders before, that's why I started this program.

Everything you asked is perfectly valid, it's really just a matter of "Is it feasible in a reasonable amount of time and work, and how much is it needed ?".
Title: Re: [TOOL] LKBC M2 Converter v0.3
Post by: sshroud on July 17, 2015, 12:33:49 am
Speaking of .mdx it could be really interesting to get to port the old WoW models (pre 1.0) to modern formats, e.g. 1.12.1 or even 3.3.5.

There are a lot of interesting examples such as the old character models, Ogres, Succubus, Infernal and the Doom Guard. Yeah, Doom Guards looked quite cool back in the early leaked alpha, like a mix between Kruul and the TBC Doom Guards, can't find any screenshots of it unfortunately.

I haven't really looked at how you did it, but knowing the data structures and the significant differences in the data is it feasible to turn it into something that we can use as a generic platform for handling conversions?
Title: Re: [TOOL] LKBC M2 Converter v0.4
Post by: spik96 on July 24, 2015, 07:52:51 pm
New Update ! Compatible with WoD character models and the program can now also convert to Classic ! (1.x)
So I guess it's LKBCCL now, but let's keep the name it will be easier.
There is even help showing in the terminal when you do something wrong !

Quote from: "sshroud"
I haven't really looked at how you did it, but knowing the data structures and the significant differences in the data is it feasible to turn it into something that we can use as a generic platform for handling conversions?

Not just the significant differences. All differences. A single thing could make the client crash and if you ever had a WoW Error you know WoW errors are NOT easy to understand (I don't at all, in fact x) ).
And that's the problem. We don't have a nice documentation about these formats. There is a big lack of knowledge.
Title: Re: [TOOL] LKBC M2 Converter v0.4
Post by: sshroud on July 24, 2015, 10:41:40 pm
Quote from: "spik96"
New Update ! Compatible with WoD character models and the program can now also convert to Classic ! (1.x)
So I guess it's LKBCCL now, but let's keep the name it will be easier.
There is even help showing in the terminal when you do something wrong !

Quote from: "sshroud"
I haven't really looked at how you did it, but knowing the data structures and the significant differences in the data is it feasible to turn it into something that we can use as a generic platform for handling conversions?

Not just the significant differences. All differences. A single thing could make the client crash and if you ever had a WoW Error you know WoW errors are NOT easy to understand (I don't at all, in fact x) ).
And that's the problem. We don't have a nice documentation about these formats. There is a big lack of knowledge.

It's certainly finicky, but don't you require full knowledge of the 3.3.5 .m2 format for writing to the 1.12.1 .m2 format? (Which could then be potentially reversible?)
Title: Re: [TOOL] LKBC M2 Converter v0.4
Post by: spik96 on July 24, 2015, 11:45:30 pm
Quote from: "sshroud"
It's certainly finicky, but don't you require to full knowledge of the 3.3.5 .m2 format for writing to the 1.12.1 .m2 format? (Which could then be potentially reversible?)

Oh 1.12.1 to 3.3.5a, sure I could (not sure if really worth it as the few good models that are not in the game anymore can be opened in MDLVis, converted to Mdx then converted using Skarn's tutorial).
But I was talking about alpha models. Those are in a special mdx format derivated from the Warcraft 3 model format and we don't have sufficient information about it, as far as I know.
You need people with retroengineering abilities (I don't have this kind of skill) to get a more complete doc. Back then nobody cared about alpha models, and nowaday they are all gone from the WoW scene. And anyway I doubt they would put so much effort (because retroengineering is really f*cking hard) just for something people will barely use.
Title: Re: [TOOL] LKBC M2 Converter v0.4
Post by: leeviathan on November 04, 2015, 03:17:09 am
(First off, thanks so much for this tool, I'm using it a lot!)

I'm trying to convert the 3.3.5a NightElfMale.M2 to 1.12.1. (Aside: I'm trying to add Barbershop hairstyles to character models in 1.12.1.) Everything seems to work fine except that the "eyeglow" geoset is missing.

Is the eyeglow considered a "light", would that be why it doesn't translate?

---------->link to my work (http://www.mediafire.com/download/8urh8yfxv2mp84o/NightElfMale.zip)
Title: Re: [TOOL] LKBC M2 Converter v0.4
Post by: leeviathan on November 10, 2015, 05:41:51 am
I'm trying to make a modification to the source in order to adjust some of the fallback animations for certain models. However, compiling the source has even my mighty software engineer friend stumped. Can you provide any guidance for compiling your source code?

thankkkkkkkkk
Title: Re: [TOOL] LKBC M2 Converter v0.4
Post by: spik96 on November 10, 2015, 11:58:08 am
I use CMake to organize a bit the code structure, but if you don't know how it works you can still take all the includes (.h files), all the source (.c files), put them in one folder then compile the main with your favorite compiler I use Linux's GCC but any converter should work.
I don't think the Night elf glow is considered a light. My converter removes the Death Knight geoset but the standard Night elf eyes should be fine. Note its texture is hardcoded, however.
Title: Re: [TOOL] LKBC M2 Converter v0.4
Post by: leeviathan on November 11, 2015, 12:40:53 am
The hardcoded blp is where it needs to be, it's just that the eye geoset is missing--I looked in WoWModelViewer and toggled all the geosets to make sure. Have a look if you get the time (I linked it in my post about the Night Elf eyes)

In trade for your attention, I humbly offer you this Tauren Bear model (https://www.mediafire.com/?wxftrrz94yff6r7). I changed a render flag set on its eye geoset, which was causing the eye to glow in the dark.
Title: Re: [TOOL] LKBC M2 Converter v0.4
Post by: spik96 on November 11, 2015, 12:31:34 pm
Interesting. I thought renderflags did not changed between the two periods. I bet it's again the default value problem.
See, before LK all model had to specify exactly their settings, but since LK, they often left a blank (under the form of an nRefs=0) and it magically works because the client knows that nothing equals a default behaviour. But when you want to convert the formats, you have to find all these blanks (so you have to know how to differentiate a 0 that means don't use that and a 0 that means use the default value) then you have to find the default value you should replace the 0 with... And that's hard.
That's even harder because to make the converter, at start I was of course looking at old models since they were on both LK and BC/Vanilla clients. But older models (still on WoD today) have still their settings entirely set and don't use default values at all.
I first found that with the attachments that were not always properly displayed and I couldn't figure why.
There was a default=true. This change was unnoticed while it's still used today (now it has been added to the wiki thanks to schlumpf) !
I will look at the renderflags. There is also one other big problem with my converter. Try to convert arctic bird. It will have no wings. The part of the geoset (and I insist on the part of the geoset and not the geoset which is also a part of the model) will disappear. This problem is also on the Jaina.m2 and all the Warlords of Draenor character models. Fixing it would allow the conversion of those problems.

Also, you had a problem with the animations of your bird. My algorithm was supposed to fix that automatically (http://hastebin.com/oberimiwap.cpp (http://hastebin.com/oberimiwap.cpp" onclick="window.open(this.href);return false;)) but it seems it's still not complete. Sorry.
Title: Re: [TOOL] LKBC M2 Converter v0.4
Post by: stan84 on November 15, 2015, 08:13:18 pm
Quote from: "spik96"
I use CMake to organize a bit the code structure, but if you don't know how it works you can still take all the includes (.h files), all the source (.c files), put them in one folder then compile the main with your favorite compiler I use Linux's GCC but any converter should work.
I don't think the Night elf glow is considered a light. My converter removes the Death Knight geoset but the standard Night elf eyes should be fine. Note its texture is hardcoded, however.

Hi Spik :)

I'd like to compile your converter,  unsucessfully :(
recevied such report:

Scanning dependencies of target UAll
[  6%] Building C object CMakeFiles/UAll.dir/tests/UDummy.c.o
[ 13%] Building C object CMakeFiles/UAll.dir/tests/CuTest.c.o
[ 20%] Building C object CMakeFiles/UAll.dir/src/conversion.c.o
[ 26%] Building C object CMakeFiles/UAll.dir/src/fallback.c.o
[ 33%] Building C object CMakeFiles/UAll.dir/src/reading.c.o
/home/develop/LKBC_Converter/src/reading.c: In function ‘read_model’:
/home/developLKBC_Converter/src/reading.c:893:20: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
    anim_files = (FILE *) fcaseopen(
                    ^
[ 40%] Building C object CMakeFiles/UAll.dir/src/common.c.o
[ 46%] Building C object CMakeFiles/UAll.dir/src/fcaseopen.c.o
Linking C executable UAll
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
collect2: error: ld returned 1 exit status
make[2]: *** [UAll] Error 1
make[1]: *** [CMakeFiles/UAll.dir/all] Error 2
make: *** [all] Error 2

could You post compiled executable, or give tip to solution ?

Stan ;)
Title: Re: [TOOL] LKBC M2 Converter v0.4
Post by: spik96 on November 16, 2015, 12:42:42 pm
The compiled version is always available in the Binaries section on Github : https://github.com/Koward/LKBC_Converter/releases (https://github.com/Koward/LKBC_Converter/releases" onclick="window.open(this.href);return false;)
Concerning the last rev, I have been told there is a bug with models where they crash the game at sight. I'm exhausted trying to fix this god damn converter. It's really painful x)

I'm considering porting this project to Java or Scala for modern and easier maintainability. And to fit the new specifications schlumpf adds from the source.
Title: Re: [TOOL] LKBC M2 Converter v0.4
Post by: stan84 on November 16, 2015, 02:24:24 pm
Thanks for the link dude.

Even now, Your tool is big step forward in tbc- vanilla modding. Keep working Spik, and don't lose heart for it.
:)
Stan