{"id":3766,"date":"2012-04-11T21:25:48","date_gmt":"2012-04-12T04:25:48","guid":{"rendered":"http:\/\/multimedia.cx\/eggs\/?p=3766"},"modified":"2012-04-27T23:19:22","modified_gmt":"2012-04-28T06:19:22","slug":"the-11th-hour-roq-variation","status":"publish","type":"post","link":"https:\/\/multimedia.cx\/eggs\/the-11th-hour-roq-variation\/","title":{"rendered":"The 11th Hour RoQ Variation"},"content":{"rendered":"

I have been looking at the RoQ file format<\/a> almost as long as I have been doing practical multimedia hacking. However, I have never figured out how the RoQ format works on The 11th Hour<\/a>, which was the game for which the RoQ format was initially developed. When I procured the game years ago<\/a>, I remember finding what appeared to be RoQ files and shoving them through the open source decoders but not getting the right images out.<\/p>\n

I decided to dust off that old copy of The 11th Hour and have another go at it.<\/p>\n


\n\"\"<\/a>
\n<\/center><\/p>\n

Baseline<\/strong>
\nThe game consists of 4 CD-ROMs. Each disc has a media\/ directory that has a series of files bearing the extension .gjd, likely the initials of one Graeme J. Devine<\/a>. These are resource files which are merely headerless concatenations of other files. Thus, at first glance, one file might appear to be a single RoQ file. So that’s the source of some of the difficulty: Sending an apparent RoQ .gjd file through a RoQ player will often cause the program to complain when it encounters the header of another RoQ file.<\/p>\n

I have uploaded some samples to the usual place.<\/strong><\/a><\/p>\n

However, even the frames that a player can decode (before encountering a file boundary within the resource file) look wrong.<\/p>\n

Investigating Codebooks Using dreamroq<\/strong>
\nI wrote dreamroq<\/a> last year– an independent RoQ playback library targeted towards embedded systems. I aimed it at a gjd file and quickly hit a codebook error.<\/p>\n

RoQ is a vector quantizer video codec that maintains a codebook of 256 2×2 pixel vectors. In the Quake III and later RoQ files, these are transported using a YUV 4:2:0 colorspace– 4 Y samples, a U sample, and a V sample to represent 4 pixels. This totals 6 bytes per vector. A RoQ codebook chunk contains a field that indicates the number of 2×2 vectors as well as the number of 4×4 vectors. The latter vectors are each comprised of 4 2×2 vectors.<\/p>\n

Thus, the total size of a codebook chunk ought to be (# of 2×2 vectors) * 6 + (# of 4×4 vectors) * 4.<\/p>\n

However, this is not the case with The 11th Hour RoQ files.<\/em><\/p>\n

Longer Codebooks And Mystery Colorspace<\/strong>
\nJuggling the numbers for a few of the codebook chunks, I empirically determined that the 2×2 vectors are represented by 10 bytes instead of 6<\/strong>. Now I need to determine what exactly these 10 bytes represent.<\/p>\n

I should note that I suspect that everything else about these files lines up with successive generations of the format. For example if a file has 640×320 resolution, that amounts to 40×20 macroblocks. dreamroq iterates through 40×20 8×8 blocks and precisely exhausts the VQ bitstream. So that all looks valid. I’m just puzzled on the codebook format.<\/p>\n

Here is an example codebook dump:
\n<\/p>\n

\r\nID 0x1002, len = 0x0000014C, args = 0x1C0D\r\n  0: 00 00 00 00 00 00 00 00 80 80\r\n  1: 08 07 00 00 1F 5B 00 00 7E 81\r\n  2: 00 00 15 0F 00 00 40 3B 7F 84\r\n  3: 00 00 00 00 3A 5F 18 13 7E 84\r\n  4: 00 00 00 00 3B 63 1B 17 7E 85\r\n  5: 18 13 00 00 3C 63 00 00 7E 88\r\n  6: 00 00 00 00 00 00 59 3B 7F 81\r\n  7: 00 00 56 23 00 00 61 2B 80 80\r\n  8: 00 00 2F 13 00 00 79 63 81 83\r\n  9: 00 00 00 00 5E 3F AC 9B 7E 81\r\n  10: 1B 17 00 00 B6 EF 77 AB 7E 85\r\n  11: 2E 43 00 00 C1 F7 75 AF 7D 88\r\n  12: 6A AB 28 5F B6 B3 8C B3 80 8A\r\n  13: 86 BF 0A 03 D5 FF 3A 5F 7C 8C\r\n  14: 00 00 9E 6B AB 97 F5 EF 7F 80\r\n  15: 86 73 C8 CB B6 B7 B7 B7 85 8B\r\n  16: 31 17 84 6B E7 EF FF FF 7E 81\r\n  17: 79 AF 3B 5F FC FF E2 FF 7D 87\r\n  18: DC FF AE EF B3 B3 B8 B3 85 8B\r\n  19: EF FF F5 FF BA B7 B6 B7 88 8B\r\n  20: F8 FF F7 FF B3 B7 B7 B7 88 8B\r\n  21: FB FF FB FF B8 B3 B4 B3 85 88\r\n  22: F7 FF F7 FF B7 B7 B9 B7 87 8B\r\n  23: FD FF FE FF B9 B7 BB B7 85 8A\r\n  24: E4 FF B7 EF FF FF FF FF 7F 83\r\n  25: FF FF AC EB FF FF FC FF 7F 83\r\n  26: CC C7 F7 FF FF FF FF FF 7F 81\r\n  27: FF FF FE FF FF FF FF FF 80 80\r\n<\/pre>\n
Note that 0x14C (the chunk size) = 332, 0x1C and 0x0D (the chunk arguments — count of 2×2 and 4×4 vectors, respectively) are 28 and 13. 28 * 10 + 13 * 4 = 332, so the numbers check out.<\/p>\n
Do you see any patterns in the codebook? Here are some things I tried:<\/p>\n