The Multimedia Guru, Michael Niedermayer, is widely known to possess an encyclopedic — and sometimes downright frightening — knowledge of multimedia technology, theory, and related mathematics. Check out this old mailing list thread, wherein we were trying to sort of the finer details of a reverse engineered, game-related video codec (Electronic Arts TQI, if you must know). Allow me to summarize:

• Reverse engineer: These floats show up in the original binary decoder and it’s anyone’s guess as to what they really mean: 1.306563, 0.541196, 0.382439.
• Michael Niedermayer: 1.3065630 = cos(pi*2/16)sqrt(2), 0.5411961 = cos(pi*6/16)sqrt(2), and “0.3824393, ROTFL, this is wrong, its certainly supposed to be: 0.3826834 (0x3EC3EF15) = cos(pi*6/16); compare: 0x3ec3cf15” (and he was right)
• Everyone else, in unison: WTF?! You knew those numbers off the top of your head?

So that pretty much left us in slack-jawed amazement. At least, until Michael revealed his secret: ‘grep -r 5411961 MPlayer’.

I can’t believe I haven’t heard of this still image format before — it’s called ART and was apparently used in earlier incarnations of America OnLine’s service. All that anyone knows about it is that it is based on compression techniques licensed from an outfit named Johnson-Grace, a firm that AOL later purchased.

The Wikipedia page implies that this format is to still image formats what Bink is to video codecs — if the marketing literature is to be believed, the compressor analyzes the data to be compressed and selects from a number of coding methods based on the assessed characteristics. The Wikipedia page also links to several J-G patents, none of which I have read.

A colleague sent me a bevy of samples for study. Where to start with a reverse engineering effort? There seems to be plenty of ancient programs that can work with the format. However, it seems reasonable that the code for decoding ART files must live on any of the billions of free AOL signup disks known to exist.

I got to see a demonstration of a restored, 45 year old DEC PDP-1 computer today at the Computer History Museum in Mountain View, CA, USA. Does that sound interesting in the context of multimedia hacking? The thing could be hooked up to some kind of dot-plotting video device, and it didn’t feature any sound audio. At least, no sound hardware out of the box. Thing is, the unit was highly mod-able.

The PDP-1 hosted what is widely believed to be the first video game ever– Spacewar!. I have already written up that aspect of the experience in my Gaming Pathology blog.

Sound, however, was possible through a hardware mod. The computer had an array of LEDs and one clever hacker thought to wire 4 of these up to square wave generators, thus producing 4-channel music. This was originally programmed in the early 1960s and was demoed today. The hacker who had originally written the music engine on a PDP-1 at MIT found himself on the restoration committee many decades later. It seems MIT had donated paper tape sequences that contained musical data that played on his music engine– but the engine code had been lost. Still, he was able to reverse engineer the audio format and reimplement the engine on the original PDP-1 hardware. Sounds familiar. He even made the same point that I like to make in my multimedia technology presentations — data is more important than code.

It almost made me feel young again. Here I am, studying multimedia formats that largely only date back about 15 years to around 1993.