Modcraft - The community dedicated to quality WoW modding!
Wrath of the Lich King Modding => Miscellaneous => Topic started by: stoneharry on September 20, 2013, 07:09:12 pm
-
I'm looking at the Blizzard downloader and I am confused as to how it functions.
There are four key directories that the Blizzard Downloader uses:
background*.torrent
updateDownloader.ini
updatePatchSequenceFile.txt
update*.mpq / exe
The downloader contains misc data:
[Downloader]
directDownloadThreshold=3500000
[DownloaderFast]
directDownloadThreshold=7000000
[BackgroundDownloader]
directDownloadThreshold=10000
[PortDefaultValues]
minPort=6881
maxPort=6999
The PatchSequenceFile points to what should be downloaded: http://us.version.worldofwarcraft.com/u ... ceFile.txt (http://us.version.worldofwarcraft.com/update/PatchSequenceFile.txt" onclick="window.open(this.href);return false;)
However this is directly contradictory to how other sources document the Blizzard Downloader to be constructed (and how I found): http://www.wowwiki.com/Blizzard_Downloader#Structure (http://www.wowwiki.com/Blizzard_Downloader#Structure" onclick="window.open(this.href);return false;)
The executable has a .torrent file attached to the end of the file which can be extracted. It also has a MPQ attached that contains all the images the downloader uses.
The torrent file can be edited using any normal default torrent editor tool. However it also appears to have some unknown data attached to the end:
(http://i.imgur.com/MUdtUv5.jpg)
This seems to contain information such as the IP and file that is allowed to download this file (directly going to the URL will tell you that you cannot access that file). MD5, and such also appears to exist. There is also an image URL that overwrites the background for the Blizzard Downloader as well as the name of the file being downloaded.
I am requesting some clarity on all of this information:
- Why there is two places the downloader can get the torrent from (URL & end of file)
- The structure of the information at the end of the torrent file
- Any other research done on the matter
With this information it should be possible to then just create a normal torrent tracker and to seed a torrent, then attach that torrent with the downloader info after to the installer executable.
-
yo,
PatchSequenceFile is used by the launcher to know wich data are needed from the server for update. its seems unlikely the downloader use this file as torrent first's purpose is especially to allow client to download from somewhere else than the server, and if a .torrent file is embedded in the .exe then it doesnt need any other informations.
As everything is done through the launcher now i cant get hold of an official downloader.
Can u upload one from 1 version to another so I can take a look a this? (english locale if u have one)
thanks
-
yo,
PatchSequenceFile is used by the launcher to know wich data are needed from the server for update. its seems unlikely the downloader use this file as torrent first's purpose is especially to allow client to download from somewhere else than the server, and if a .torrent file is embedded in the .exe then it doesnt need any other informations.
As everything is done through the launcher now i cant get hold of an official downloader.
Can u upload one from 1 version to another so I can take a look a this? (english locale if u have one)
thanks
Sure, if you login through WoW instead of launcher or download a game (not a patch) they still use the downloader system. I think the PatchSequenceFile was used on a different version of the Blizzard Downloader that didn't have the torrent attached - this is from a long time ago when I got it working before. This version seems to be more up to date and a better system.
Here is the Downloader I used to download WC3 from Blizzard last week, it is restricted to my IP as stated in the thread: https://dl.dropboxusercontent.com/u/110 ... s_enGB.zip (https://dl.dropboxusercontent.com/u/1102355/Downloader_Warcraft3_Reign_of_Chaos_enGB.zip" onclick="window.open(this.href);return false;)
I haven't been able to do much testing yet as I have been moving houses which is a long and tiresome process. Once I am set up again I will begin doing some testing. I want to finish getting the Battle.net 2 protocol implemented (which I am half done on), the blizz downloader working, and then work on the Launcher. :)
Thanks.
-
Okay, I replaced the torrent file & unknown data with my own torrent file and attempted to test it that way. I just get the error message:
"There is nothing to download"
Now, so the unknown data is essential it seems in this version of the Blizz Downloader, or I am doing something very wrong.
-
the end of the torrent is at offset 0xD51A on ur screenshot.
unknown info are a list of struct similar to each blizzard files but they are written in reverse order :
- block data
- block header
header is as follow :
struct
{
char Name[4];
signed int Offset; // offset of data from beggining of header (always negative)
};
so u see the unknown data contains several block of data :
RESP ( contain torrent file data? )
COKI ( list of key/values quite explicit. u can deduce the informations easily : ip, expires, access, md5)
PANE ( url of the image displayed )
MODE ( unknown 1 unsigned integer = 0x00000001)
there is one more block before torrent data named SKIN wich contain and MPQ file (holding ui data)
however... this downloader dont just download a file, it ask to select a folder, and probably do much more things. It seems much more complex than downloaders we used to obtain from Wow for update download and i suspect it to be notably different of the one used for updates.
What do u plan to do with the downloader? if u want to use it for update, i suggest u use an update downloader as its seems more generic (guess they didnt made code change from one update to another, just torrent change)
-
What do u plan to do with the downloader? if u want to use it for update, i suggest u use an update downloader as its seems more generic (guess they didnt made code change from one update to another, just torrent change)
Thanks for the information.
Yeah I will try and use a different version of the downloader. :)
edit:
I downloaded the downloader from 3.3.5a -> 4.x.x by logging into retail. After removing the torrent they have attached and attaching my one I get 'There is nothing to download' still. There did not appear to be any structure after the torrent file which confuses me.
-
as said earlier, u can change the torrent file but u still have to add the header right after. on some downloader the COKI, PANE and MODE chunks are missing. leaving only torrent data, and skin data.
here is the utility i made for extracting/appending new torrent files (work on windows) http://www.filesplat.com/.happyhack/sha ... zDLasm.rar (http://www.filesplat.com/.happyhack/shared/V2UHNRMBT9AXEVLB9JHE5Q2GATJ06GFY----BlizzDLasm.rar" onclick="window.open(this.href);return false;)
BlizzDLasm.exe extract "example.downloader.exe" for extracting the torrent file
BlizzDLasm.exe compile "example.torrent" "out.exe" for creating a new downloader with the given torrent file
however when i change the torrent for another, the downloader seems to work correctly but dont download anything.
i found some informations about the downloader there http://www.networkuptime.com/wow/page02-09.html (http://www.networkuptime.com/wow/page02-09.html" onclick="window.open(this.href);return false;)
Its said the downloader connect to blizzard's tracker.
Maybe the downloader try to connect to blizzard's tracker automatically and stop downloading if the torrent informations are not found... maybe blizzard's tracker is more than a simple torrent tracker and it send informations needed by the downloader to correctly complete the download process.... maybe it just check if the torrent's tracker match blizzard's one...
maybe i just did something wrong... I m afraid the downloader need informations only delivered by blizzard's tracker.
my knowledge in reversing are quite limited so i can do nothing more, sorry
-
Interesting, thanks for the information.
It seems there are many different versions of the downloader. I have got it to work before a couple of years ago, 100%. I didn't need to attach a header and I believe it got the torrent file from a URL rather than attached. I'm going to see if I can find this version.
edit:
There is a BackgroundDownloader.exe inside the MPQs. I am going to do some experimenting with this version since it is shipped with the game.
-
This thread seems to have got it a version working as well: viewtopic.php?p=16499#p16499 (http://modcraft.io/viewtopic.php?p=16499#p16499" onclick="window.open(this.href);return false;)
This thread also explains it: viewtopic.php?p=1309#p1309 (http://modcraft.io/viewtopic.php?p=1309#p1309" onclick="window.open(this.href);return false;)
Also my post here from a while ago shows how I got it working: viewtopic.php?p=14826#p14826 (http://modcraft.io/viewtopic.php?p=14826#p14826" onclick="window.open(this.href);return false;) (though I cannot remember doing this!)
I am simply unable to find the strings that point to the directory containing these folders now. :/ No log file is being created when I run the program either, only "Data for the next patch is unavailable."
-
This thread seems to have got it a version working as well: viewtopic.php?p=16499#p16499
haha these screenshots :lol:
on my side i finally make it work
here are all explanation :
you have to create a torrent file with :
- the announce field set to your tracker url (during my test i set it to http://127.0.0.1:3724/announce (http://127.0.0.1:3724/announce" onclick="window.open(this.href);return false;) it just worked well)
several field are used by the downloader for additionnal informations (choose output path, launch executable after full download, etc...)
Blizzard's torrent tracker is customized so when u send a torrent request, the response will have additionnals informations : a http URL for direct download.
then, the downloader try to download from the direct http URL it just got from the tracker.
The file on the http server should be splitted in several files to match the "piece length" field in the torrent file.
the downloader try to download from the direct http url like that :
"http://direct_http_url/piece_nbr"
during my test i set the direct http url to "http://127.0.0.1/WoW-3.3.5.12213-to-3.3.5.12340-frFR-patch.exe"
so the downloader tried to access http://127.0.0.1/WoW-3.3.5.12213-to-3.3 ... atch.exe/0 (http://127.0.0.1/WoW-3.3.5.12213-to-3.3.5.12340-frFR-patch.exe/0" onclick="window.open(this.href);return false;)
then http://127.0.0.1/WoW-3.3.5.12213-to-3.3 ... atch.exe/1 (http://127.0.0.1/WoW-3.3.5.12213-to-3.3.5.12340-frFR-patch.exe/1" onclick="window.open(this.href);return false;), etc...
here is a very tiny bittorrent tracker that does the job, and a splitter for the downloaded file, both written in c#.
http://www.filesplat.com/.happyhack/sha ... racker.rar (http://www.filesplat.com/.happyhack/shared/46CECYJAUA1P2TA69YPP9IJ94P9P3JSF----TinyTracker.rar" onclick="window.open(this.href);return false;)
(u should not use this tracker for a working server. It does not implement the torrent protocol fully, it just does the job for this test)
a bittorrent tracker is very simple, i guess u could even make one with a single php script on a http server.
whether u choose to use one or another is up to you. I didnt read the topics u linked in ur last post in detail but it seems an easier solution. However if u want to do it the same way blizzard does, then here it is ;)
tell me if u need any help
-
Quite some magic Internet points for you, happyhack.
-
...
Brilliant, thank you!
I set up a torrent file using uTorrent. I set the piece size to 1024 KB, added a HTTP link to the file, and set the tracker to localhost.
I replaced this torrent with the one in WoW-x.x.x.x-4.0.0.12911-Downloader (enUS) which is sent when logging into battle.net in build 12340 (3.3.5a).
I then added the downloader info such as Target, locale, etc as you have done. I changed the string lengths as appropriate.
I split the target file into 1024 KB chunks using the tool you provided.
The torrent program you wrote opens on the specified port correctly.
Now when I try to run the downloader, I am getting the error message "There is nothing to download!".
When I run the Blizzard one, I get the same error message for some strange reason. It should work as it is the one sent from Blizzard.
I have no idea what could be going wrong.
(http://i.imgur.com/iaVG54N.jpg)
I was thinking it might be because the torrent created by uTorrent is not in the format that the Downloader requires but it doesn't answer why the original unedited version does not work.
-
Quite some magic Internet points for you, happyhack.
behind a router 8-)
Now when I try to run the downloader, I am getting the error message "There is nothing to download!".
i see on ur screenshot the headers are not set correctly. I know its confusing because what I called header is at the END of each chunk, not at the beggining.
here is the utility i made for extracting/appending new torrent files (work on windows) http://www.filesplat.com/.happyhack/sha ... zDLasm.rar (http://www.filesplat.com/.happyhack/shared/V2UHNRMBT9AXEVLB9JHE5Q2GATJ06GFY----BlizzDLasm.rar" onclick="window.open(this.href);return false;)
BlizzDLasm.exe extract "example.downloader.exe" for extracting the torrent file
BlizzDLasm.exe compile "example.torrent" "out.exe" for creating a new downloader with the given torrent file
plz use this utility, it does all the job for you and create a working downloader from a torrent file. (its yet in the last download in the folder "torrent + downloader" under the name "bwod.exe")
-
Ah yes, it works great! :) Excellent.
It's connecting to the tracker fine and the filesplitter program makes sense.
The direct download URL is sent in the tracker response, however my downloader does not appear to be using it. At least, it's not downloading from it. The filesplitter would only be used for sending using the torrent protocol, correct?
-
The filesplitter would only be used for sending using the torrent protocol, correct?
correct, almost :D actually peers send pieces to each others trough TCP, not HTTP (if i remember well). But you are right, the downloader get same pieces from the http server than it would get from a peer.
The direct download URL is sent in the tracker response, however my downloader does not appear to be using it
hmmm... does the downloader gives u any error or warning message?
are you sure the pieces are available to the downloader? can u download them through a browser?
Does the View > Connection info menu show u the correct url in the "Direct HTTP download" row?
-
The filesplitter would only be used for sending using the torrent protocol, correct?
correct, almost :D actually peers send pieces to each others trough TCP, not HTTP (if i remember well). But you are right, the downloader get same pieces from the http server than it would get from a peer.
The direct download URL is sent in the tracker response, however my downloader does not appear to be using it
hmmm... does the downloader gives u any error or warning message?
are you sure the pieces are available to the downloader? can u download them through a browser?
Does the View > Connection info menu show u the correct url in the "Direct HTTP download" row?
Okay, I'm a bit confused as to what I should point to.
(http://i.imgur.com/thHNEba.jpg)
At the moment I just have the download URL set to the binary file on a web server.
The parts (0 to 29) and the hash.txt is uploaded in the same directory as the binary file.
-
Wow.exe should be a folder, not a file.
Delete Wow.exe, create a directory with that name, then place all the part files inside. it should work now.
u can remove hash.txt too, it is unused.
-
Wow.exe should be a folder, not a file.
Delete Wow.exe, create a directory with that name, then place all the part files inside. it should work now.
u can remove hash.txt too, it is unused.
That makes more sense. :) It now starts to download but then it stops and a bad.piece is created at the download location.
The torrent piece size is set to 256 kb. That's 262144 bytes. So I am splitting the file using filesplitter @ 262144 bytes.
When the download stops the Wow.exe.part has reached 768 KB. The bad.part is 255 KB (261,396 bytes).
Since it seems to fail almost immediately and due to the size of the bad.part I think this is because one of the bits that the filesplitter has created is an incorrect size. So I recreated the parts and uploaded the new ones to the same result. Am I splitting it by the wrong value? Should it be 256KB - 1?
-
Since it seems to fail almost immediately and due to the size of the bad.part I think this is because one of the bits that the filesplitter has created is an incorrect size. So I recreated the parts and uploaded the new ones to the same result. Am I splitting it by the wrong value? Should it be 256KB - 1?
no. filesplitter is correct, it writes in the console how many bytes are written to each file, u can check it by hand too.
The part size doesnt matter, i tried with 256, 512, 1024, it worked just well.
In the torrent file there is a field "pieces" which contain concatened sha1 hash of all pieces. If µtorrent didnt set this correctly the download will fail because of piece integrity check.
(this is what the hash.txt file is used for)
If u change the pieces size without updating the torrent, the download will fail because of piece integrity check.
If u resume the download without delete "yourfiledownloaded.exe.part" after changing pieces size, the download will fail because of piece integrity check.
-
Since it seems to fail almost immediately and due to the size of the bad.part I think this is because one of the bits that the filesplitter has created is an incorrect size. So I recreated the parts and uploaded the new ones to the same result. Am I splitting it by the wrong value? Should it be 256KB - 1?
no. filesplitter is correct, it writes in the console how many bytes are written to each file, u can check it by hand too.
The part size doesnt matter, i tried with 256, 512, 1024, it worked just well.
In the torrent file there is a field "pieces" which contain concatened sha1 hash of all pieces. If µtorrent didnt set this correctly the download will fail because of piece integrity check.
(this is what the hash.txt file is used for)
If u change the pieces size without updating the torrent, the download will fail because of piece integrity check.
If u resume the download without delete "yourfiledownloaded.exe.part" after changing pieces size, the download will fail because of piece integrity check.
I see. You are correct, it seems uTorrent does not populate the hash. I google'd a bit on how to fill this field and came up with info_hash being the required field. The hash.txt is 1200 in length. So I added it like so:
d8:announce31:http://127.0.0.1:54177/announce9:info_hash1200:[...]10:created by13:uTorrent/331013:creation datei1380975771e8:encoding5:UTF-84:infod6:lengthi7704216e4:name7:Wow.exe12:piece lengthi262144e6:pieces600
Where [...] = the contents of hash.txt.
However it is still failing to get any piece correctly. Is this the correct field? Do you recommend another program for generating torrent files?
Thanks for helping me with what are probably silly mistakes. :P
-
d8:announce31:http://127.0.0.1:54177/announce9:info_hash1200:[...]10:created by13:uTorrent/331013:creation datei1380975771e8:encoding5:UTF-84:infod6:lengthi7704216e4:name7:Wow.exe12:piece lengthi262144e6:pieces600
wont work, hash.txt is a hex representation of binary data.
i use bencode editor, write 0x[...] in value field to make it binary
-
So I took a step back and had a look at the Blizzard torrent files to see how it should be fully constructed. I can now see that my torrent files I was creating were not even similar to the structure Blizzard used.
So I took a Blizzard torrent and populated it with the data for my file:
(http://i.imgur.com/m3Qgmyu.jpg)
I am assuming pieces is the hash as it seems like the logical place for it.
Now when I run it I still get a bad.piece generated next to the Blizzard Downloader and a UpdatesWow.exe.part filled with some data.
If this appears to all be correct and is just a incorrect value somewhere then I will continue testing different things.
-
i dont know... there is a working torrent file in the folder "torrent + downloader".
If it doesnt work then u are messing somewhere. Read again the entire topic. If it still doesnt work then.... read again the entire topic, then read it a 3rd time.
If it still doesnt work then upload ur files.
Its a complex process to make a working downloader this way. I told u the tracker program is really bad, if u want to make it work fully u need to make one yourself or modify an existing bittorrent tracker.
There can be many differents problems, u have to find the solution by urself.
-
Hi there!
First of all I love your guys and all your work, you got all my respect.
I'm noob in network stuff, happyhack you said:
I told u the tracker program is really bad, if u want to make it work fully u need to make one yourself or modify an existing bittorrent tracker.
Can you tell me what is missing in the tinytracker you made? (split peers var for multiple download?)
What do i risk using it for a working server?
Or can you give me a link or a name of an easy/simple opensource tracker that I can use for a working server?
Thanks in advance
Edit:
https://github.com/Hlkz/peertracker/com ... 266b8e762f (https://github.com/Hlkz/peertracker/commit/08a296eda63958177b6b343ca009a6266b8e762f" onclick="window.open(this.href);return false;)
And it seems working for now.
-
Hy sorry for resurrecting necro thread , is possible to upload again the files because is not working anymore :( . This one : BlizzDLasm.rar