Modcraft - The community dedicated to quality WoW modding!
Wrath of the Lich King Modding => Miscellaneous => Topic started by: stoneharry on March 26, 2013, 08:08:28 pm
-
I'm working on replicating the full Blizzard patching process, whereby your login to WoW and a patch is sent to you containing a downloader, the downloader downloads a installer that then installs the patch and starts WoW back up updated.
With the help of Schlumpf, all but the last stage has been achieved.
The Blizzard Installer is made up of a series of parts in a single file:
- The main installer program
- An attached MPQ archive
- BsnI data (installer data)
- Signature data
Schlumpf wrote a tool to extract the base program from it, and then to be able to assemble a patch onto the installer program with the necessary BsnI data.
However, both the assembler and extractor through exceptions when I try to use them. Inflation is failing. For example, in the extractor, this line of code:
int result = inflateInit (&stream); // this returns -2
if (result)
{
return result;
}
Returns -2 which throws an exception that inflation failed.
The source and cmake files can be found here: https://dl.dropbox.com/u/1102355/BsnI.zip (https://dl.dropbox.com/u/1102355/BsnI.zip" onclick="window.open(this.href);return false;)
The full function that this error is returned from is shown below:
int wrap_inflate ( unsigned char* output, int size_output
, const unsigned char* input, int size_input
, int* written_output
)
{
z_stream stream;
stream.zalloc = NULL;
stream.zfree = NULL;
stream.next_in = const_cast<Bytef*> (input);
stream.avail_in = size_input;
stream.next_out = output;
stream.avail_out = size_output;
int result = inflateInit (&stream); // this returns -2
if (result)
{
return result;
}
int inflateResult = inflate (&stream, 4);
if (inflateResult == 1)
{
*written_output = stream.total_out;
return inflateEnd (&stream);
}
else
{
inflateEnd (&stream);
if (inflateResult != 2 && (inflateResult != -5 || stream.avail_in))
{
return inflateResult;
}
return -3;
}
}
If anyone is able to reproduce and fix this, then it would be greatly appreciated. I have no idea what could be wrong. I have tested it with the 3.0.1 to 3.0.2 installer and the 3.3.5 to 3.3.5a installer.
-
#define Z_STREAM_ERROR (-2)inflateInit returns Z_OK if success, Z_MEM_ERROR if there was not enough memory, Z_VERSION_ERROR if the zlib library version is incompatible with the version assumed by the caller, or Z_STREAM_ERROR if the parameters are invalid, such as a null pointer to the structure. msg is set to null if there is no error message. inflateInit does not perform any decompression apart from possibly reading the zlib header if present: actual decompression will be done by inflate(). (So next_in and avail_in may be modified, but next_out and avail_out are unused and unchanged.) The current implementation of inflateInit() does not process any header information -- that is deferred until inflate() is called.
-
(http://i.imgur.com/EZ8efYH.jpg)
All the data seems to be set. Output has no contents but it is initialised. I do not understand why it would think the parameters are invalid.
Unless next_in and input are the issues (next_in is input casted).
Input is: &(*output)[0], and output is a empty unsigned character vector.
Tracing this back:
std::vector<unsigned char> output_data (entry->inflated_data_size);
zlib::inflate (input_data, &output_data);
It seems output_data is still the same empty vector.
So I am confused as to why it would consider any of these parameters invalid.
-
output_data is supposed to be empty, but the exact size given in the BsnI. Input should not be &(*output)[0] though, but &input[0]. Oo
-
output_data is supposed to be empty, but the exact size given in the BsnI. Input should not be &(*output)[0] though, but &input[0]. Oo
Yeah, it is correct - I was wrong. I misread it.
int wrap_inflate ( unsigned char* output, int size_output
, const unsigned char* input, int size_input
, int* written_output
)
Being called by:
const int res (wrap_inflate (&(*output)[0], output->size(), &input[0], input.size(), &inflated));
-
Version of zlib?
-
Version of zlib?
#define ZLIB_VERSION "1.2.3"
#define ZLIB_VERNUM 0x1230
Edit: Updated my zlib binary in the executable directory, now result is returning -3. Code:
int wrap_inflate ( unsigned char* output, int size_output
, const unsigned char* input, int size_input
, int* written_output
)
{
z_stream stream;
stream.zalloc = NULL;
stream.zfree = NULL;
stream.next_in = const_cast<Bytef*> (input);
stream.avail_in = size_input;
stream.next_out = output;
stream.avail_out = size_output;
int result = inflateInit (&stream);
if (result)
{
return result;
}
int inflateResult = inflate (&stream, 4);
std::cout << std::endl << "Inflate result from inflate(&stream, 4) = " << inflateResult << std::endl;
if (inflateResult == 1)
{
*written_output = stream.total_out;
return inflateEnd (&stream);
}
else
{
inflateEnd (&stream);
std::cout << "if inflateResult != 2 && (inflateResult != 5 || stream.availin) return result, else return -3" << std::endl;
std::cout << "stream.avail_in: " << stream.avail_in;
if (inflateResult != 2 && (inflateResult != -5 || stream.avail_in))
{
return inflateResult;
}
return -3; // this is being returned
}
}
Result:
(http://i.imgur.com/gtLxYtZ.jpg)
-
#define Z_BUF_ERROR (-5)
Z_BUF_ERROR if no progress is possible (for example avail_in or avail_out was zero).
stream.avail_in = size_input;
stream.avail_out = size_output;
Both size_input and size_output have a value.
stream.avail_in is 0.
stream.avail_out is 12499.
So because avail_in is zero, that could trigger this error. However, I don't know why this would be the case. It becomes 0 after:
int inflateResult = inflate (&stream, 4);
-
Why is avail_in 0? It should be about half of avail_out, as given in the BsnI entry.
-
Why is avail_in 0? It should be about half of avail_out, as given in the BsnI entry.
It is initially this:
(http://i.imgur.com/HL9T4Q8.jpg)
Then it becomes 0 after:
inflateInit (&stream);
However it is not half of avail_out in the first place.
-
Being half was only an estimation. Any non-zero number will do.
Given that total_in/out are not the same as avail_in/out: This really is the state before calling anything in zlib?
-
Being half was only an estimation. Any non-zero number will do.
Given that total_in/out are not the same as avail_in/out: This really is the state before calling anything in zlib?
Yes, it is the initial state.
Main calls extract_entry (input_file, &entry);
extract_entry calls zlib::inflate (input_data, &output_data);
zlib::inflate calls wrap_inflate (&(*output)[0], output->size(), &input[0], input.size(), &inflated)
wrap_inflate executes:
int wrap_inflate ( unsigned char* output, int size_output
, const unsigned char* input, int size_input
, int* written_output
)
{
z_stream stream;
stream.zalloc = NULL;
stream.zfree = NULL;
stream.next_in = const_cast<Bytef*> (input);
stream.avail_in = size_input;
stream.next_out = output;
stream.avail_out = size_output;
Then I breakpoint and prevent further execution. That is when the screenshot shows.
After calling the next line:
int result = inflateInit (&stream);
avail_in becomes 0.
-
In your screenshot, zalloc and zfree are non NULL. It is technically impossible that this is the state at that line.
-
In your screenshot, zalloc and zfree are non NULL. It is technically impossible that this is the state at that line.
I don't know why, but that is the case:
(http://i.imgur.com/VGxrtj7.jpg)
http://i.imgur.com/VGxrtj7.jpg (http://i.imgur.com/VGxrtj7.jpg" onclick="window.open(this.href);return false;)
edit: Pretty sure that's just showing the memory address, and it has a value of NULL still.
-
The addresses would be closer together. Also, it would not be logical at all, to display the address of anything: It is a void* = NULL. There is no more address than the position of the variable itself (on the stack) and the value NULL.
-
The addresses would be closer together. Also, it would not be logical at all, to display the address of anything: It is a void* = NULL. There is no more address than the position of the variable itself (on the stack) and the value NULL.
Okay. However I don't understand why it would have a value, if that is the case.
-
Try printf("zalloc:: %pn", stream.zalloc); instead of that debugger.
-
Try printf("zalloc:: %pn", stream.zalloc); instead of that debugger.
(http://i.imgur.com/aGszxOB.jpg)
-
Well, I guess all values are wrong then.
-
Well, I guess all values are wrong then.
The pointers for all the variables set in the stream, not that it is particularly useful other than proving they have been set:
(http://i.imgur.com/BN5L3MT.jpg)
Still no idea why it does not work.
-
Attempted with a few different patches, they all produce the same results. I can see why -5 is being returned (because the other value becomes 0 after the first inflation) but I have no idea why this is happening or how you would go about debugging and fixing this.
Any suggestions?
-
Avoiding the extractor for now, the assembler appears to work.
Using the example_base I extracted for contents of the 3.3.5 to 3.3.5a patch and create a new MPQ from that.
I then am trying to assemble a new executable using this data and the example base, but the arguments required confuse me.
(arguments: <base> <output> <mpq> <exe> [<files...>])
Base = example_base.exe
Output = A new file? (I put test.exe)
MPQ = the MPQ to be attached
exe = ??? I put the original installer in here
<Files> = ??? Additional optional files to be included not in the MPQ?
Either way, the executable being produced (after no errors) does not run and throws a Windows error about it not being a valid windows application. So I assume I am getting the arguments wrong.
-
example_base.exe output.exe patch.mpq exe_extracted_from_original_installer.exe dlls_extracted_from_original_installer.dll
-
example_base.exe output.exe patch.mpq exe_extracted_from_original_installer.exe dlls_extracted_from_original_installer.dll
Ah, so the extractor needs to be functioning correctly first.
-
I don't see any reason, why it would not work, anyway. oO
-
Updated zlib to 1.2.7 and converted project to visual studio 2011/12 for a more up to date debugger.
Before inflation:
(http://i.imgur.com/WmDwmd7.jpg)
After inflation:
(http://i.imgur.com/Cdm2w7W.jpg)
Key things to note is how inflateResult goes from uninitialised to -5. avail_out goes from a large value to 0.
The returned value of -5 signals a error, usually caused by avail_out or avail_in becoming 0, as it has in this case.
As to why it happens, I have had no idea.
I do remember when you posted in the shoutbox about it before you finding a typo. Has that been fixed in this version? I do not remember what it was.
-
Has that been fixed in this version?
How would I know which "version" you're using?!
Try doubling the size allocated for output.
-
Has that been fixed in this version?
How would I know which "version" you're using?!
Try doubling the size allocated for output.
I am using whatever version you gave me when I requested to start again a while ago. :P So I dunno'.
Avail_out starts at 6052592 and goes to 3026284.
Avail_in starts at 1499174 and goes to 0.
So now -5 is still being returned, but this time because avail_in is 0 rather than avail_out.
6052592 / 2 =
3026296 which is just bigger than the original value if not / 2:
3026284
-
If avail_in is 0, it worked?
-
If avail_in is 0, it worked?
No, because -5 is being returned it throws an error that inflation failed.
-
Note that Z_BUF_ERROR is not fatal,
It just means, that either there is no more output or no more input. Either, it is perfectly fine and inflated everything or the input is incomplete. As output begins with 'MZ', the output is also likely to be correct.
-
And given, that not Z_STREAM_END is returned, it is likely that the input has stopped too early.
-
Well, if I trace your code through showing only the lines of code executed:
int inflateResult = inflate (&stream, 4);
...
inflateEnd (&stream);
...
return -3;
-> returned
if (res || inflated != output->size())
{
throw std::runtime_error ("error: not inflated correctly");
}
So, if your saying it should still have worked, shall I change the code so that it continues?
edit:
I assume it should be doing this:
if (inflateResult == 1)
{
*written_output = stream.total_out;
return inflateEnd (&stream);
}
Where, ideally 1 is returned (success?) so the written_output then gets a value (inflated), otherwise inflated will not match the output size.
-
Try writing output to disk and look at it. Also, try increasing input size read from BsnI. I'd wonder if, but it may be wrong.
The last possibly wrong thing is the size of unsigned int being wrong, but I really can't believe that.
-
Try writing output to disk and look at it. Also, try increasing input size read from BsnI. I'd wonder if, but it may be wrong.
The last possibly wrong thing is the size of unsigned int being wrong, but I really can't believe that.
Well I modified all the error handling so that it writes it, and it resulted in three files being output:
Installer.exe
RichEd20.dll
Unicows.dll
Which looks correct to me. o_0
-
The filenames of course are fine. The question is if the contents are..
-
The filenames of course are fine. The question is if the contents are..
Apparently not:
(http://i.imgur.com/a4DnrBk.jpg)
-
try increasing input size read from BsnI.
Where is this done?
-
The only thing I'd suspect being wrong is the compression mode / strength chosen for inflate. But as you know, I successfully extracted your installer using exactly that source code.
May you give sizeof(BsnI_Entry) (or whatever it was called)?
-
The only thing I'd suspect being wrong is the compression mode / strength chosen for inflate. But as you know, I successfully extracted your installer using exactly that source code.
May you give sizeof(BsnI_Entry) (or whatever it was called)?
sizeof(BsnI); returns 16.
sizeof(BsnI_entry); returns 76.
-
Those values are fine.
I have no idea what might be wrong, then. It should work fine.
Try different numbers for inflate, try using bsnientry.data_size * 2.
-
Those values are fine.
I have no idea what might be wrong, then. It should work fine.
Try different numbers for inflate, try using bsnientry.data_size * 2.
When I did that, the returned result from inflation was 1 rather than -5, which signals a success. :)
However, when assembling using those output files, it still does not work. The file size was larger.
-
Why try assembling them? The exe is a executable itself and should be possible to run.
-
Why try assembling them? The exe is a executable itself and should be possible to run.
Ah, didn't realise this. However, the extraction must still not be correct because:
(http://i.imgur.com/NmVzB9K.jpg)
-
Use 1.2.3. I guess other versions than their own won't get proper results.
-
Use 1.2.3. I guess other versions than their own won't get proper results.
Tested with 1.2.3 version of zlib again.
Modifications to code:
input_file.read_into (&input_data, entry->data_size * 2);
std::vector<unsigned char> output_data (entry->inflated_data_size * 2);
Both values are * 2.
The inflation always returns 1 (success).
The inflated integer is: 3026296
The output size is: 6052592 (6052592 / 2 = 3026296), which matches the inflated integer. So modified the condition that checks for errors to:
if (res || inflated != output->size() / 2)So everything is inflated correctly and no errors are generated throughout.
However the resulting executable file cannot be run - the same error message as which I previously posted:
(http://i.imgur.com/NmVzB9K.jpg)
I believe the executable file has not been constructed correctly, since it cannot be opened with a archive program as any other Windows executable can be.
-
I had someone else test this and they had the same issues with as me, both before and after the modifications. Did you test this on Windows?
-
Of course I did not.
-
Hey guys,
im interested in this too,
here is the error (should add a binary flag in open_flag enum instead) :
// /! BINARY MODE !!!!!!
static const char* to_string (const open_flag& flag)
{
if (flag == read)
{
return "rb";
}
else if (flag == write)
{
return "wb";
}
else if (flag == (read|write))
{
return "rwb";
}
else
{
throw std::runtime_error ("bad opening flags");
}
}
(win32 > filestream :P )
hope this can help
btw nice work, tried to reverse the installer a long time ago but didnt manage to get it work !
-
Gnah. m(
-
void seek (const int& pos) const
{
if (pos <= 0)
{
fseek (_handle, -pos, SEEK_END);
}
else
{
fseek (_handle, pos, SEEK_SET);
}
}
should be pos < 0 else seek(0) result in EOF ( seek(0) called in read_all() )
in append.bsni.cpp,
BsnI_entry* entry (reinterpret_cast<BsnI_entry*> (&entries[i]));
should be
BsnI_entry* entry (reinterpret_cast<BsnI_entry*> (&entries[sizeof(BsnI_entry)*i]));
because entries is an array of char
then all is working fine.
But each time I try to assemble the installer, I get the same message
"version 3.3.5.12213 is needed and you already have version 3.3.5.12340", regardless which version of the extracted installer I repack in, or which mpq I use.
-
void seek (const int& pos) const
{
if (pos <= 0)
{
fseek (_handle, -pos, SEEK_END);
}
else
{
fseek (_handle, pos, SEEK_SET);
}
}
should be pos < 0 else seek(0) result in EOF ( seek(0) called in read_all() )
in append.bsni.cpp,
BsnI_entry* entry (reinterpret_cast<BsnI_entry*> (&entries[i]));
should be
BsnI_entry* entry (reinterpret_cast<BsnI_entry*> (&entries[sizeof(BsnI_entry)*i]));
because entries is an array of char
then all is working fine.
But each time I try to assemble the installer, I get the same message
"version 3.3.5.12213 is needed and you already have version 3.3.5.12340", regardless which version of the extracted installer I repack in, or which mpq I use.
Thanks for the information, I will test this when I have some free time. :)
The attached MPQ needs to be made in a special way - it shouldn't just be a random MPQ you want to install.
Open one of Blizzards patches with a MPQ editor (e.g: 3.3.5 to 3.3.5a) and then look at the files. There are instructions on which files to extract/execute/install/move, need to experiment. There is one file that contains installer information - the target WoW version and what version it patches to, etc.
I would test this by taking the contents of 3.3.5 to 3.3.5a and changing the target version and contents that is copied over.
edit: Just made your changes and it was extracted perfectly! :) Will do more tests later. I have also updated the original download link on the first post of the thread - it now contains the original version (that worked on Mac) and the Windows version (the changes you have submitted).
edit two: Currently only able to generate the generic error "Out of harddrive space". This error is standard - it does not mean what it says. Will need to trace it in ollydebug to see why it is failing - at what condition. I expect it is due to Blizzard's signature not being found or similar since they put this verification on all the other parts.
-
Bringing everything together at the moment, just need to get the launcher and this fully working (viewtopic.php?f=26&t=4379 (http://modcraft.io/viewtopic.php?f=26&t=4379" onclick="window.open(this.href);return false;)) then push it into a single tidy document.
On the subject of this - the Blizzard Installer - I have no idea where to go next. In the contents of the attached MPQ is a file size - patch.cmd "PatchSize 1992294400" - what should this value be representing? The file size of the MPQ or the entire installer? Even when this is correct (for each value), I cannot get it to start up correctly.
Why is there a Blizzard Updater.exe within the contents of a patch? What is this for? (Referencing 3.3.5 to 3.3.5a).
Is there any verification methods Blizzard uses that I need to disable?
-
hmmmm... dont know what I did wrong during my last post but nvm, it seems everything work fine.
The .cmd file in the mpq contains information on the version of the updater.
In the contents of the attached MPQ is a file size - patch.cmd "PatchSize 1992294400" - what should this value be representing?
was wondering too... seems a lot too large for a file size... but the updater work anyway.
dont know what you could do wrong... i got the "disk full" message sometimes because I closed the repacker before full writing so the "BsnI" section was missing. You have to extract an mpq from an installer correctly too.
I have also updated the original download link on the first post of the thread - it now contains the original version (that worked on Mac)
u should make the modifications to this one too....
Text files are files containing sequences of lines of text. Depending on the environment where the application runs, some special character conversion may occur in input/output operations in text mode to adapt them to a system-specific text file format. Although on some environments no conversions occur and both text files and binary files are treated the same way, using the appropriate mode improves portability.
opening a file with "fopen("rw")" open the file as a text file... on binary file this may result in corrupted data. Maybe on mac it will work, but if there is an error someday, it will be seriously hard to find.
-
hmmmm... dont know what I did wrong during my last post but nvm, it seems everything work fine.
The .cmd file in the mpq contains information on the version of the updater.
In the contents of the attached MPQ is a file size - patch.cmd "PatchSize 1992294400" - what should this value be representing?
was wondering too... seems a lot too large for a file size... but the updater work anyway.
dont know what you could do wrong... i got the "disk full" message sometimes because I closed the repacker before full writing so the "BsnI" section was missing. You have to extract an mpq from an installer correctly too.
I have also updated the original download link on the first post of the thread - it now contains the original version (that worked on Mac)
u should make the modifications to this one too....
Text files are files containing sequences of lines of text. Depending on the environment where the application runs, some special character conversion may occur in input/output operations in text mode to adapt them to a system-specific text file format. Although on some environments no conversions occur and both text files and binary files are treated the same way, using the appropriate mode improves portability.
opening a file with "fopen("rw")" open the file as a text file... on binary file this may result in corrupted data. Maybe on mac it will work, but if there is an error someday, it will be seriously hard to find.
Ah, okay. Thank you for the clarifications.
dont know what you could do wrong... i got the "disk full" message sometimes because I closed the repacker before full writing so the "BsnI" section was missing. You have to extract an mpq from an installer correctly too.
What do you mean by correctly? I'm opening it up with a default MPQ editor and extracting it that way at the moment. Is there another way you should do it?
And I open the assembler through command prompt, it executes fine from the looks of it:
(http://i.imgur.com/6rUQnJb.jpg)
-
The bad flags on opening the file were my error. On (li|u)n(i|u)x, it seems fine as no conversion takes place. On windows, line feeds get converted.
The other corrections are fine as well, they all had been an error on my side. Please remove my bad code from the download.
PatchSize equals 1,8GB, thus hardly is too large. I think it is the maximum size per patch-#.MPQ, to avoid FAT32 limitations.
The BsnI chunk contains filenames, which are copied, as given in the command line. Thus, your updater will extract to C:UsersHarryDesktopBsnI Project, thus will break upon execution due to not finding files.
-
Thank you, schlumpf. Now it is executing correctly and progress is being made. The information you have given also explains a lot. :)
Progress:
World of Warcraft Update (3.3.5.12340 - 3.3.5.12341)
This patch upgrades World of Warcraft from version 3.3.5.12340 to version 3.3.5.12341.
Installing World of Warcraft Update at "C:WoW 3.3.5" on Tue Apr 02 17:58:41 2013.
Error: The file "C:WoW 3.3.5output.exe" appears to be corrupt. You may need to download this file again.
Things I don't understand:
How does it know which patch to write to? Patch.lst obviously contains the path in the MPQ, file name, path to write to.
Delete.lst contains files to delete at start of the patch (?? repeated file names in existing patches (realmlist.wtf appears 4 times in 3.0.2 to 3.3.5a))
Other list files? Verification files to look for?
Blizzard Updater.exe? What is this for? Does this contain the information on what to write to?
I'm being told my installer is corrupt. How is it determining this? I removed all the instructions other than to move battle.net.dll and to go from 12340 to 12341.
-
Things I don't understand:
Blizzard Updater.exe? What is this for? Does this contain the information on what to write to?
It applies the downloaded patches, so I'd imagine, yes it contains the info on what to write to.
There used to be a manual patch workaround to bypass the launcher so you don't auto-update past the version you want...
Basically you take the MPQ and TFIL Files that match your locale and OS...
Drop them in your wow directory... Rename the MPQ to 'wow-patch.MPQ'
Then run the updater, and it takes the wow-patch.MPQ info and updates to that patch for you.
--Update--
Found a more recent link to the above explanation... Maybe this guy outlined it better....
http://eu.battle.net/wow/en/forum/topic/1710171900 (http://eu.battle.net/wow/en/forum/topic/1710171900" onclick="window.open(this.href);return false;)
-
How does it know which patch to write to? Patch.lst obviously contains the path in the MPQ, file name, path to write to.
there is a "WoWPatchIndex 3" in patch.cmd, the number change from one update to another. Maybe it has something to do with it.
Things I don't understand:
Blizzard Updater.exe? What is this for? Does this contain the information on what to write to?
It applies the downloaded patches, so I'd imagine
hmmm.... yes. I saw it in some old updates. But it seems its not used in theses... i think u can delete it. This exe is generic. The one from 11723-to-12213.exe and 12213-to-12340.exe are strictly identicals.
I'm being told my installer is corrupt. How is it determining this? I removed all the instructions other than to move battle . net . dll and to go from 12340 to 12341.
because of strong signature check.
The signature check is done in Installer.exe and u can bypass it by replacing 0x75 by 0xEB at file offset 0x15E2ED in the installer.exe extracted from WoW-3.3.5.12213-to-3.3.5.12340-xxXX-patch.exe
The way the installer check the version of Wow.exe is through the resource version data of the exe. u can change it with a resource editor/hacker.
Finally, .exe inside of the .mpq are not working exe, they seems to be bsdiff patch, have to confirm this too.
-
there is a "WoWPatchIndex 3" in patch.cmd, the number change from one update to another. Maybe it has something to do with it.
Yeah, I guessed this would be the case - however some patches do not specify this - and what about where multiple patches are written to? :)
I've modified the version info and component data in the WoW executable now and will test the Installer executable edits shortly. Thank you for the help.
edit:
Worked, thank you:
(http://i.imgur.com/VtyuUIe.jpgp)
Almost fully working:
(http://i.imgur.com/MDDjGkH.jpg)
edit2: That index number does determine which file to write to, I got "writing to patch-4.mpq" before it errors on me. ;)
edit3:
(http://i.imgur.com/4bOWPgr.jpg)
Getting there.
edit4:
At the moment, as soon as it tries to write some data it crashes. I'm testing with a single blp file being written to patch-4.MPQ and tried testing with a random file. Not sure why yet, now to try with 3.3.5 to 3.3.5a contents again but tell it to write to 4 instead of 3.
edit5:
The patch "basecomponent.wow-data.txt" could not be applied. (Invalid header: size 25404, version 111.) Patch can't be applied because it is missing data for: "base".It seems these component files are vital in same way.
-
Copied the 3.3.5 to 3.3.5a installer contents.
Changed everything from 3.3.5 to 3.3.5a inside (components, etc).
Replaced battle.net.dll and WoW.exe and runonce.wtf with own values.
Changed patch index to 4.
When I assemble then run:
The update could not be applied.
Blizzard Updater was unable to read the file "C:WoW 3.3.5DataenGBbackup-enGB.MPQ". This error may be caused by problems with the media or drive at C:--for example, a scratched or dirty CD-ROM/DVD-ROM, hard drive corruption, or a networking problem while downloading the update. (The data being read was "Battle.net.dll", and the error code was 0.) If this problem persists, you may be able to solve it by uninstalling and then reinstalling the game. If you are unable to correct this problem, please contact Blizzard Technical Support. (Converter::Load)
To check this installation for problems, click the "Repair" button. The Repair tool can automatically fix many update errors.
Not figured what could cause this yet. If I remove the battle.net.dll installation from the patch.lst, then I get this error:
The update could not be applied.
Blizzard Updater was unable to read the file "C:WoW 3.3.5DataenGBbackup-enGB.MPQ". This error may be caused by problems with the media or drive at C:--for example, a scratched or dirty CD-ROM/DVD-ROM, hard drive corruption, or a networking problem while downloading the update. (The data being read was "Wow.exe", and the error code was 0.) If this problem persists, you may be able to solve it by uninstalling and then reinstalling the game. If you are unable to correct this problem, please contact Blizzard Technical Support. (Converter::Load)
To check this installation for problems, click the "Repair" button. The Repair tool can automatically fix many update errors.
So then it is doing the same error with the Wow.exe.
After removing all the base moves from patch.lst, it seems it has the same error with every file:
The update could not be applied.
Blizzard Updater was unable to read the file "C:WoW 3.3.5DataenGBbackup-enGB.MPQ". This error may be caused by problems with the media or drive at C:--for example, a scratched or dirty CD-ROM/DVD-ROM, hard drive corruption, or a networking problem while downloading the update. (The data being read was "DataenGBDocumentationReadMeEULA.html", and the error code was 0.) If this problem persists, you may be able to solve it by uninstalling and then reinstalling the game. If you are unable to correct this problem, please contact Blizzard Technical Support. (Converter::Load)
To check this installation for problems, click the "Repair" button. The Repair tool can automatically fix many update errors.
I removed the backup MPQ from my WoW directory and ran it again:
The update could not be applied.
The patch "pc-game-hdfilesWTFRunOnce.wtf" could not be applied. (Invalid header: size 17747, version 84.) If this problem persists, you may be able to solve it by uninstalling and then reinstalling the game. If you are unable to correct this problem, please contact Blizzard Technical Support. (BNUpdate::PTCApply)
To check this installation for problems, click the "Repair" button. The Repair tool can automatically fix many update errors.
It seems there is some sort of verification upon each file inside the patch or something?
-
Blizzard Updater was unable to read the file "C:WoW 3.3.5DataenGBbackup-enGB.MPQ".
i never got that error, u did something wrong.
The patch "pc-game-hdfilesWTFRunOnce.wtf" could not be applied. (Invalid header: size 17747, version 84.)
Actually, here is the problem :
"base\component.wow-data.txt" and all "component.*.txt" files have extra data at the beginning of the file. A kind of header 24 bytes large (not sure if 24).
My mpq editor doesnt export this header.... strange, but as I export it, the resulting file just contain the text data.
(Try to export "base\component.wow-data.txt" and, then import it back in the .mpq, then rebuild the updater and run it, u will probably get an error)
But reading the file with stormlib library, I get all the data. Then if I replace the file then now run the updater, it will work.
Actually, i think all file except these in the main folder ("hdfiles.lst", "delete.lst", "patch.cmd", ect...) have this header.
.exe inside of the .mpq are not working exe, they seems to be bsdiff patch, have to confirm this too.
All file, except "component.*.txt" and the files in the main folder seems to be bsdiff patches. That mean u cannot just add some file to the .mpq and hope that work. It won't. u should make a bsdiff patch from this file, then add the header at the beginning.
The header is something like this :
edit: updated
struct UpdateFileHeader
{
UINT16 HdrSize; // always 0x0018
UINT8 version; // always 0x04
UINT8 override; // 0x01 => override with new file; 0x04 => update file (bsdiff)
UINT32 crc32; // of the file before update
UINT32 OldFileSize; // file size before the update (uncompressed)
UINT32 NewFileSize; // size of the entire file after the update is applied (uncompressed)
UINT32 unknown6;
UINT32 unknown7;
};
-
It should not need to be a bsdiff. Iirc, if the installer sees a bsdiff-header, it automatically applies it as patch. If it does not, it should just overwrite the file.
-
It should not need to be a bsdiff. Iirc, if the installer sees a bsdiff-header, it automatically applies it as patch. If it does not, it should just overwrite the file.
right, can add a normal file (not bsdiff patch), but need to modify the header accordingly.
Header updated.
-
Hacked together a little program to copy the contents of the extracted patch to a new folder, with the header applied for every file:
http://pastebin.com/byTHJB2i (http://pastebin.com/byTHJB2i" onclick="window.open(this.href);return false;)
I'm not very good at C++. :) At the moment folder structure is not maintained, and old file size I left as 0.
(http://i.imgur.com/8zbQA30.jpg)
Whoop whoop!
This patch upgrades World of Warcraft from version 3.3.5.12340 to version 3.3.5.12341.
Installing World of Warcraft Update at "C:WoW 3.3.5" on Sat Apr 06 14:40:29 2013.
Created folder "C:WoW 3.3.5Cache".
Created folder "C:WoW 3.3.5CacheWDB".
Created folder "C:WoW 3.3.5CacheWDBenGB".
Created folder "C:WoW 3.3.5Logs".
Created file "C:WoW 3.3.5DataenGBbackup-enGB.MPQ".
Created file "C:WoW 3.3.5DataenGBpatch-enGB-4.MPQ".
Created file "C:WoW 3.3.5Datapatch-4.MPQ".
Created file "C:WoW 3.3.5LogsBlizzard Updater Log.html".
Created file "C:WoW 3.3.5Patch.html".
Created file "C:WoW 3.3.5Patch.txt".
Created file "C:WoW 3.3.5WTFRunOnce.wtf".
Created file "C:WoW 3.3.5Battle.net.dll".
Created file "C:WoW 3.3.5Wow.exe".
<Win32_UnableToAddWorld of Warcraft UpdateToGamesExplorer>
The update was successful.
edit: Updated my pastebin link.
I'm getting:
The patch "pc-battlenet-hdfilesBattle.net.dll" could not be applied. (Invalid header: size 24, version 1.)
When I specify overwrite instead of 0x04 (backup first), as backup needs the original actual file size.
My header is:
18 00 01 04 00 00 00 00 00 00 00 00 80 DB ED 00 00 00 00 00 00 00 00 00
Which looks correct to me.
When I set it to 0x04 but set the old file size as the same as the current file size (which it should be since both my Battle.net.dll's are exactly the same file size:
The patch "pc-battlenet-hdfilesBattle.net.dll" could not be applied. (CRC mismatch: expected 0x00000000, actual 0xb58429b0.
-
error from me, sorry. Updated
-
Updated my messy code from the pastebin link.
I am getting CRC mismatch still:
The patch "pc-battlenet-hdfilesBattle.net.dll" could not be applied. (CRC mismatch: expected 0x00000000, actual 0xb58429b0.)
I am guessing that the header is saying it should be 0 and the other is the actual value retrieved.
The CRC returned from battle.net.dll in my function is:
3045337520
So I don't know why it is seeing 0 unless I am writing the header in the wrong order.
I set unknown 6 and 7 to crc just to see what would happen:
The patch "pc-battlenet-hdfilesBattle.net.dll" could not be applied. (PTCApply_BSDiff flunked.)
So I set override to 0x01 instead of 0x04 and the patching process fully worked! o_0 So unknown 6 or 7 must be the CRC not unknown 3.
So this is the code where it all works: http://pastebin.com/yuLAHvsK (http://pastebin.com/yuLAHvsK" onclick="window.open(this.href);return false;)
-
Yup, it's all working perfectly now. :) Ran many tests and compiled a 5000+ file installer that runs successfully.
Note that empty files will cause the installer to throw an error. I'm not sure if this is because the header becomes corrupt because the file size is empty or what, but I had to remove a few empty files from my Glue and Frame XML folders.
Cheers, I'll try to type up the entire process clearly and formally at some point.
-
Great work guys, this is a huge step forward in providing an A++ custom server experience :)
-
So unknown 6 or 7 must be the CRC not unknown 3.
no no... ^^ header is correct
hdr => 18 00 04 04 01 00 00 00 80 DB ED 00 80 DB ED 00 02 00 00 00 03 00 00 00
error => CRC mismatch: expected 0x00000001, actual 0x0527e254
So I set override to 0x01 instead of 0x04 and the patching process fully worked!
yes, its ok.
It should not need to be a bsdiff.
battle.net.dll is not a bsdiff patch. Its a fully working dll, i was wrong at 1st.
In its original header its override field was set to 0x01. If u extract files from an update .mpq, be sure to keep the header unharmed.
Note that the crc32 field is only checked if override field is set to 0x04.
Still the last 2 field of the header are unknown... be carefull about that.
I'm not very good at C++.
OutFile.write(reinterpret_cast<const char*>(&test.HdrSize), sizeof(UINT16));
OutFile.write(reinterpret_cast<const char*>(&test.version), sizeof(UINT8));
OutFile.write(reinterpret_cast<const char*>(&test.override), sizeof(UINT8));
OutFile.write(reinterpret_cast<const char*>(&test.crc32), sizeof(UINT32));
OutFile.write(reinterpret_cast<const char*>(&test.OldFileSize), sizeof(UINT32));
OutFile.write(reinterpret_cast<const char*>(&test.NewFileSize), sizeof(UINT32));
OutFile.write(reinterpret_cast<const char*>(&test.crc32), sizeof(UINT32));
OutFile.write(reinterpret_cast<const char*>(&test.crc32), sizeof(UINT32));
:lol: (no offend ^^)
OutFile.write(reinterpret_cast<const char*>(&test), sizeof(UpdateFileHeader));
;)
Great work guys, this is a huge step forward in providing an A++ custom server experience
yep good job about the extractor wich was the most of the work.
-
Hello guys maybe can create a little tutorial for newcomers?
-
Hello guys maybe can create a little tutorial for newcomers?
No not really. I was going to make a tutorial that shows all the steps and documents each part but I ran out of free time and it won't exactly be fun to do.
If you get stuck you can ask questions here. If you don't know where to start, maybe this is too advanced for you. Try reading through the pages of this thread.
-
The error i came when i try to install modified update from 335 to 335a to patch to 336.12341. When i open 335a update with ladik mpqeditor i update files and when i open appended installer and open i see installer data. What is wrong. I try to open my test updater i run to this:(
The file "pc-game-hdfilesWow.exe" could not be opened. If this problem persists, you may be able to solve it by uninstalling and then reinstalling the game. If you are unable to correct this problem, please contact Blizzard Technical Support. (InstallerFile::Open)
-
The error i came when i try to install modified update from 335 to 335a to patch to 336.12341. When i open 335a update with ladik mpqeditor i update files and when i open appended installer and open i see installer data. What is wrong. I try to open my test updater i run to this:(
The file "pc-game-hdfilesWow.exe" could not be opened. If this problem persists, you may be able to solve it by uninstalling and then reinstalling the game. If you are unable to correct this problem, please contact Blizzard Technical Support. (InstallerFile::Open)
Not sure what you are trying to say because your English is poor (sorry!).
The primary steps are:
- Extract contents of a old patch (I used 3.3.5 to 3.3.5a)
- Modify contents to your patch
- Update the patch.lst file
- Write the installer header binary data to the beginning of every file in the base folder and the WoW binary and the battle.net.dll if you are writing that and the WTF run once file.
- Create a MPQ from this data.
- Extract installer data.
- Use append to take installer data and the MPQ created to write a new executable installer.
-
it is normal to have size of wow executable around 24bytes?
-
it is normal to have size of wow executable around 24bytes?
No. :) You wrote just the header and not the original data.
For me:
Before header:
7,704,216 bytes.
After header:
7,704,240 bytes.
File size difference = 24 bytes.
Header:
(http://i.imgur.com/GegeyTY.png)
-
thank stoneharry i achieved that thing. I append with example_base.exe as base and is working like intended.
Edit :
component-*.txt need to have the header?
-
thank stoneharry i achieved that thing. I append with example_base.exe as base and is working like intended.
Edit :
component-*.txt need to have the header?
I am not sure what the component file is for - it seems to have no effect no matter what it says. It does need to have a header if you are including it.
One thing I have discovered is that an AVI file (movie) does need a header to install but the installer doesn't strip the header when copying it into the MPQ for some reason. If anyone knows how to solve this it would be appreciated.