Here’s a mild multimedia-related reverse engineering challenge for you. It’s pretty straightforward for those skilled in the art.
The Setup
One side effect of running this ridiculously niche interest blog at the intersection of multimedia, reverse engineering, and game hacking is that people occasionally contact me for assistance on those very matters. So it was when one of my MobyGames peers asked if I can help to extract some music from a game called Aztec Wars. The game consists of 2 discs, each with a music.xbe file that contains multiple tunes and is hundreds of megabytes large.
That’s all the data I received from the first email. At first I’m wondering what makes people think I have some magical insight into cracking these formats with such little information. Ordinarily, I would need to have the entire data file to work with and possibly the game binaries. But I didn’t want to ask him to upload hundreds of megabytes of data and I didn’t feel like downloading it; commitment issues and all.
But then I gathered a little confidence and remembered that the .xbe files are probably just Game Resource Archive Formats (GRAF) which are, traditionally, absurdly simple. I asked my colleague to send me a hexdump of the first kilobyte of one of the .xbe GRAFs ('hexdump -C -n 1024 music.xbe > file') as well as the total file size of the GRAF.
The Hexdump
The first music.xbe file is 192817376 bytes large. These are the first 1024 144 bytes (more than enough):
00000000 01 00 00 00 60 04 00 00 14 00 00 00 01 00 00 00 |....`...........| 00000010 0d 00 00 00 48 00 00 00 94 39 63 01 1c a4 21 03 |....H....9c..¤!.| 00000020 7a d2 54 04 04 28 ad 05 d8 88 fd 06 d8 88 fd 06 |zÒT..(.Ø.ý.Ø.ý.| 00000030 2a 6e 46 08 2a 6e 46 08 2a 6e 46 08 2a 6e 46 08 |*nF.*nF.*nF.*nF.| 00000040 50 13 2f 0a e0 28 7e 0b 52 49 46 46 44 39 63 01 |P./.à(~.RIFFD9c.| 00000050 57 41 56 45 66 6d 74 20 10 00 00 00 01 00 02 00 |WAVEfmt ........| 00000060 44 ac 00 00 10 b1 02 00 04 00 10 00 64 61 74 61 |D¬...±......data| 00000070 fc 13 63 01 00 00 00 00 00 00 00 00 00 00 00 00 |ü.c.............| 00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
The Challenge
Armed with only the information in the foregoing section, figure out a method for extracting all the audio files in that file and advise on their playback/conversion. Ideally, this method should require minimal effort from both you and the person on the other end of the conversation.
The Resolution
The reason I ask is because I came up with a solution but knew, deep down, that there must be a slightly easier way. How would you solve this?
The music files in question are now preserved on YouTube (until they see fit to remove them for one reason or another).
1) those are only first 144 bytes but that’s enough
2) stored format is obvious and any game ripper would extract audio (with no conversion needed)
For those who wonder, at offset 0x10 there are number of entries in the archive (litte-endian) and then 32-bit offsets to the entries follow.
@Kostya: Thanks for keeping me honest.
My specific solution to this problem was to manually type out a bunch of ‘dd’ commands that my colleague was able to copy and paste into a Unix terminal and extract each file. The reason it was tedious was because I had to sort out the start offset and file sizes in decimal (since ‘dd’ refuses to accept hex values).
Generic data recovery tools like TestDisk should be able to find ordinary wav like this within a file.
So there comes the old rule in: it saves time to look at existing stuff/tools before embarking on reverse-engineering :-)
Though I did write my own once, extractor-gtk on sourceforge, but for this case it would be overkill.
@Reimar: Thanks for the tip on TestDisk. It’s always good to learn new tools. I used to want to write all my own tools. Now I just want results. :-) I’ll be sure to play around with TestDisk.
Another approach I was thinking of was to write a simple program (like so many I have had to write before) that would manually crawl through the file and sniff out the WAV RIFF headers, sanity check the header fields, and copy out what appeared to be valid WAV files. I could have tested that locally on a contrived case and sent it along to him.
Another issue that occurred to me: Why did they have to go the uncompressed PCM route? I wonder if they could have jammed all the data on 1 disc instead of 2 if they had employed, e.g., Vorbis compression (this was a 2002 game).
“Another issue that occurred to me: Why did they have to go the uncompressed PCM route? I wonder if they could have jammed all the data on 1 disc instead of 2 if they had employed, e.g., Vorbis compression (this was a 2002 game).”
1999, actually.
I’m guessing the date confusion comes from MobyGames’s idiotic tendency to display, on the game’s main page, the date the game was released in your country (2002 in USA) and not the date it was released originally.
Few possible reasons.
If you believe in lazy developers: Because, to my knowledge, DirectSound can play wav files directly, with a single function call, including looping an other stuff.
More likely: Game was release in 1999, and due to being a budget title was designed to run on 4 year old PCs at least (i.e. anything that shipped with Windows 95 at least).
Which means: the kind of PC that can’t do much when it plays MP3 and probably can’t play Vorbis at all. (Note: MMX was a 1996 technology).
Yeah, I took another look at the game’s tech specs as listed on MobyGames and saw that the game required a Pentium II. Since these tracks sound like the background music during the gameplay, decoding perceptual audio would not have left much time for anything else.
Still, IMA ADPCM or something. I’m kind of a compression bore, I know. :-)