Category Archives: xbox

Xbox Sphinx Protocol

I’ve gone down the rabbit hole of trying to read the Xbox DVD drive from Linux. Honestly, I’m trying to remember why I even care at this point. Perhaps it’s just my metagame of trying to understand how games and related technologies operate. In my last post of the matter, I determined that it is possible to hook an Xbox drive up to a PC using a standard 40-pin IDE interface and read data sectors. However, I learned that just because the Xbox optical drive is reading an Xbox disc, that doesn’t mean it’s just going to read the sectors in response to a host request.

Oh goodness, no. The drive is going to make the host work for those sectors.

To help understand the concept of locked/unlocked sectors on an Xbox disc, I offer this simplistic diagram:

Any DVD drive (including the Xbox drive) is free to read those first 6992 sectors (about 14 MB of data) which just contain a short DVD video asking the user to insert the disc into a proper Xbox console. Reading the remaining sectors involves performing a sequence of SCSI commands that I have taken to calling the “Sphinx Protocol” for reasons I will explain later in this post.

References
Doing a little Googling after my last post on the matter produced this site hosting deep, technical Xbox information. It even has a page about exactly what I am trying to achieve: Use an Xbox DVD Drive in Your PC. The page provides a tool named dvdunlocker written by “The Specialist” to perform the necessary unlocking. The archive includes a compiled Windows binary as well as its source code. The source code is written in Delphi Pascal and leverages Windows SCSI APIs. Still, it is well commented and provides a roadmap, which I will try to describe in this post.

Sphinx Protocol
Here is a rough flowchart of the steps that are (probably) involved in the unlocking of those remaining sectors. I reverse engineered this based on the Pascal tool described in the previous section. Disclaimer: at the time of this writing, I haven’t tested all of the steps due to some Linux kernel problems, described later.

Continue reading →

Interfacing to an Xbox Optical Drive

The next generation Xbox is going to hit the streets soon. But for some reason, I’m still interested in the previous generation’s unit (i.e., the original Xbox). Specifically, I’ve always wondered if it’s possible to use the original Xbox’s optical drive in order to read Xbox discs from Linux. I was never curious enough to actually buy an Xbox just to find out but I eventually came across a cast-off console on a recycle pile.

I have long known that the Xbox has what appears to be a more or less standard optical drive with a 40-pin IDE connector. The only difference is the power adapter which I surmise is probably the easiest way to turn a bit of standardized hardware into a bit of proprietary hardware. The IDE and power connectors look like this:

Thus, I wanted to try opening an Xbox and plugging the optical drive into a regular PC, albeit one that supports IDE cables, and allow the Xbox to supply power to the drive. Do you still have hardware laying around that has 40-pin IDE connectors? I guess my Mac Mini PPC fits the bill, but I’ll be darned if I’m going to pry that thing open again. I have another IDE-capable machine buried in my closet, last called into service when I needed a computer with a native RS-232 port 3 years ago. The ordeal surrounding making this old computer useful right now can be another post entirely.

Here’s what the monstrosity looks like thanks to characteristically short IDE cable lengths:

Xbox optical drive connected directly to PC

Click for larger image

Process:

Turn on Xbox first
Turn on PC

Doing these things in the opposite order won’t work since the kernel really wants to see the drive when booting up. Inspecting the 'dmesg' log afterward reveals interesting items:
hdd: PHILIPS XBOX DVD DRIVE, ATAPI CD/DVD-ROM drive hdd: host max PIO5 wanted PIO255(auto-tune) selected PIO4 hdd: UDMA/33 mode selected [...] hdd: ATAPI DVD-ROM drive, 128kB Cache

Why is that interesting? When is the last time to saw disk devices prefixed by ‘hd’ rather than ‘sd’? Blast from the past. Oh, and the optical drive’s vendor string clearly indicates that this is an Xbox drive saying ‘hi!’.

Time To Read
When I first studied an Xbox disc in a normal optical drive, I noticed that I was able to read 6992 2048-byte sectors — about 14 MB of data — as reported by the disc table of contents (TOC). This is just enough data to play a standard DVD video animation that kindly instructs the viewer to please use a proper Xbox. At this point, I estimated that there must be something special about Xbox optical drive firmware that knows how to read alternate information on these discs and access further sectors.

I ran my TOC query tool with an Xbox Magazine demo disc in the optical drive and it reported substantially more than 6992 sectors, enough to account for more than 2 GB of data. That’s promising. I then tried running 'dd' against the device and it was able to read… about 14 MB, an exact quantity of bytes that, when divided by 2048 bytes/sector, yields 6992 sectors.

Future (Past?) Work
Assuming Google is your primary window into the broader internet, the world is beginning to lose its memory of things pertaining to the original Xbox (Microsoft’s naming scheme certainly doesn’t help searches). What I’m saying is that it can be difficult to find information about this stuff now. However, I was able to learn that a host needs to perform a sort of cryptographic handshake with the drive at the SCSI level before it is allowed to access the forbidden areas of the disc. I think. I’m still investigating this and will hopefully post more soon.

xbfuse

Well, you knew this was coming– xbfuse. This is a program that leverages the wonderful FUSE paradigm to mount a Microsoft Xbox disc filesystem — the so-called XDVD filesystem — under Linux. I hammered out the bug mentioned in yesterday’s post (sure enough, a 64-bit offset was being demoted to a signed 32-bit quantity at one point, and that matters for filesystems this large). This is what the program looks like in action:

$ xbfuse Halo-3.iso mnt/

$ ls -al mnt/
total 4
dr-xr-xr-x  6 melanson melanson       0 2007-11-07 20:00 .
drwxr-xr-x 47 melanson melanson    4096 2007-11-10 17:31 ..
dr-xr-xr-x  2 melanson melanson       0 2007-11-07 20:00 bink
-r--r--r--  1 melanson melanson 8929280 2007-11-07 20:00 default.xex
dr-xr-xr-x  5 melanson melanson       0 2007-11-07 20:00 maps
dr-xr-xr-x  2 melanson melanson       0 2007-11-07 20:00 $SystemUpdate
dr-xr-xr-x  2 melanson melanson       0 2007-11-07 20:00 waves
-r--r--r--  1 melanson melanson  561152 2007-11-07 20:00 WaveShell-Xbox.dll
-r--r--r--  1 melanson melanson  724992 2007-11-07 20:00 WavesLibDLL.dll

$ ls -al mnt/bink/
total 0
dr-xr-xr-x 2 melanson melanson        0 2007-11-07 20:00 .
dr-xr-xr-x 6 melanson melanson        0 2007-11-07 20:00 ..
-r--r--r-- 1 melanson melanson 77940860 2007-11-07 20:00 attract_1_60.bik
-r--r--r-- 1 melanson melanson 61324440 2007-11-07 20:00 attract_2_60.bik
-r--r--r-- 1 melanson melanson 72829508 2007-11-07 20:00 attract_3_60.bik
-r--r--r-- 1 melanson melanson 69631000 2007-11-07 20:00 credits_60.bik
-r--r--r-- 1 melanson melanson 21163412 2007-11-07 20:00 intro_60.bik

$ fusermount -u mnt/

So Halo 3 uses Bink files, some very high resolution ones, rather than any Xbox-specific multimedia formats, like XMV. Actually, Bungie (the company behind Halo) may have a history with Bink, as I seem to recall that the FMV for the PC demo version of Halo was also Bink (or at least one promotional file).

I actually just thought to look up whether there are other options for mounting Xbox filesystem images under Linux. The format certainly seems to be of much greater interest than, say, GameCube filesystem images. I did find a project called Mount ISO Image that is supposed to be able to handle XDVD filesystems. Though I can’t really figure out if it’s a KDE application, a script, or a KDE script.

I took a slightly different approach to writing this one. All in all, I suppose the result is much simpler than gcfuse. The GameCube filesystem is an odd beast and required a lot of custom hacks to parse all of the data structures. However, writing xbfuse scared me more because I had to write 2 mutually recursive functions: After loading the volume descriptor, call xbfs_recurse_directory(), which then calls xbfs_recurse_file_subtree(), which calls not only itself, but also xbfs_recurse_directory() when a file entry happens to be a directory. I thought about writing xbfuse in such a way that it would traverse the data structures on demand when loading a file, since the data structures are laid out to be conducive to binary searching. I also thought about only loading the first level of the directory tree, and loading other levels on demand. But in the end, I just went with the full tree load at the outset and finally squashed the 64 -> 32-bit bug and the program seems to work quite well.

What next? Wii discs seem to use a different format than the GameCube discs and I would like to find out what that is. Plus, I am still dogged by the slightly custom Dreamcast ISO-9660-style format. There is a lot of interesting Sofdec media on those Dreamcast games. And it only takes about 26 hours to rip the contents of a Dreamcast disc onto your PC, provided that you have the right serial cable.

Related Posts:

Xbox Multimedia, Finally!

At long last, I wrote that FUSE driver for the Microsoft Xbox DVD filesystem. I will hopefully get around to releasing it, just as soon as I can test it a little bit more. I am not completely confident with the new driver but that might be because the filesystem that I was using to test the program appears to have a number of problems and might be corrupted (or it could very well be that Xbox 360 games use a slightly different filesystem layout than Xbox games, or that I’m not correctly handling 32-bit thresholds on large filesystem images). When I started studying other smaller Xbox filesystems, things seemed to go quite smoothly.

So did I finally find some samples of the fabled XMV format? Yes! At long last. Far and away, however, the Bink format appears to dominate, at least on the sample of games that I surveyed.

I found 4 Xbox games that include XMV media: Thrillville, Xiaolin Showdown, Monopoly Party, and Lego Star Wars 2: The Original Trilogy. You can find samples at the usual place. The last game on that list surprised me– Lego Star Wars is an aggressively multi-platform franchise. While the disc only encodes the intro movie as an XMV file (and in both PAL and NTSC variations), this would imply to me that versions for other platforms are encoded in that platform’s preferred SDK format, i.e., MPEG-2 for PS2, THP or H4M for GameCube, and who knows for Windows. Typically, cross-platform games take the easy way out using portable middleware solutions like Bink.

A little bird once told me that XMV was on the horizon for Xbox multimedia and that it would be straight ASF files with WMV2 video and MS ADPCM audio. If that’s true, then it looks like the WMV2 puzzle was solved just in time. These XMV files certainly are not stock ASF files. Curiously, the most notable signature is ‘Xbox’ spelled, ahem, in little endian notation in bytes 12-15: ‘xobX’. Width and height are encoded at bytes 20-23 and 24-27, respectively. I see sample rate data at bytes 40-43. I tried to find framerate data by comparing the headers of the PAL vs. NTSC Lego SW demo movies. I was hoping to find fields with 50 or 25 for PAL and 60 or 30 for NTSC. No such luck. However, I have also read that the movies are 29.97 fps, so the framerate data might be encoded as floating- or fixed-point numbers.

The Lego Star Wars 2 game has lots of interesting stuff to peruse. Since it is LucasArts software, it should be no surprise that human-readable scripts play a role. Indeed, there is a whole directory of scripts, such as this simpler file scripts/Attack.scp:

state Base {
        Conditions {
                if GotOpponent == 1 goto StartAttack
        }
        Actions {
                BreakFormation
        }
}

state StartAttack {
        Conditions {
        }
        Actions {
                KeepWeaponOut
                SetState "Attack"
        }
}

state Attack {
        Conditions {
                if GotOpponent == 0 goto Base
        }
        Actions {
                EngageOpponent "goalrange 1.5" "firerange 3"
        }
}

Thrillville is also from LucasArts and has various markings to indicate that the Lua language is involved, as in other LucasArts titles such as Grim Fandango.

Lego Star Wars 2 also contains a large number of .wavm files. These could be headerless PCM or ADPCM data. What was that FFmpeg incantation for converting headerless, raw PCM to a container format, manually specifying parameters?

One more item– Call of Duty 3. This uses a format with the extension XBV and it begins with the signature ‘AFMV’. I don’t have many other clues on it, but I have thrown together a MultimediaWiki page on the matter and uploaded some samples, as is customary.