This past week, the internet picked up — and subsequently sprinted like a cheetah with — an unsourced and highly unsubstantiated rumor that Google will open source the VP8 video codec, recently procured through their On2 acquisition. I wager that the FSF is already working on their press release claiming full credit should this actually come to pass. I still retain my “I’ll believe it when I see it” attitude. However, I thought this would be a good opportunity to consolidate all of the public knowledge regarding On2′s VP8 codec.

Pictured: All the proof you need that VP8 is superior to H.264
Update: The preceding comment is meant in sarcastic jest. Read on

The Official VP8 Facts:
Read the rest of this entry »

I have been reading way too many statements from people who confidently assert that Google will open source all of On2′s IP based on no more evidence than… the fact that they really, really hope it happens. Meanwhile, I have found myself pettily hoping it doesn’t happen simply due to the knowledge that the FSF will claim total credit for such a development (don’t believe me? They already claim credit for Apple dropping DRM from music purchases: “Our Defective by Design campaign has a successful history of targeting Apple over its DRM policies… and under the pressure Steve Jobs dropped DRM on music.”)

But for the sake of discussion, let’s run with the idea: Let’s assume that Google open sources any of On2′s intellectual property. Be advised that if you’re the type who believes that all engineering problems large and small can be solved by applying, not thought, but a mystical, nebulous force called “open source”, you can go ahead and skip this post.

The Stack

Read the rest of this entry »

I’ve been hearing it ever since last August:

Google owns On2. They are going to open source all of On2′s codecs.
Read the rest of this entry »

I read something in the past few months which noted that, in this day and age, the ultimate phase of any tech startup’s business plan is to be purchased by Google. Viewed through that lens, On2 is about to live the dream, even though they existed years before Google, years before most people even knew what a search engine was.

So Google has announced its intention to purchase On2. Wow, it feels like the end of an era. It seems like I’ve had some relationship with On2 for the entire 9 years I’ve been into multimedia hacking. Something that got lost in yesterday’s coverage and commentary was that On2 started life as the Duck Corporation, also a codec company. During this period, they largely focused on gaming applications. But I’m pretty sure that RAD Game Tools kicked them out of that market with their Smacker and Bink technologies. However, files encoded with the Duck’s multimedia codecs were among the first I studied back around 2000-2001. So that always makes me sentimental.

Read the rest of this entry »

Have you heard of Sun’s JavaFX? It’s due out later this year and is allegedly positioned to compete in the RIA space. It might be pertinent to mention that I work on a competing technology. Anyway, the reason I bring this up is that I recently learned that On2 is reported to be supplying JavaFX with video codec technology. According to “Sun Adds Comprehensive Video Capabilities to Ubiquitous Java Platform with On2 Technologies,” Sun licensed On2′s “TrueMotion” codec. I’m not entirely sure what codec they’re talking about and I can’t quite find any solid details. On2′s site seems to classify TrueMotion as encompassing both VP6 and VP7. I’m always surprised to hear the name TrueMotion since I thought that went away after the Duck Corporation morphed into On2. But the VP* series seems to be interchangeable with TrueMotion, just for extra confusion.

Who knows? Maybe they actually are using the classic Duck TrueMotion video codec in JavaFX.

Curiously, there is no word on what JavaFX will use for audio. Maybe logarithmic PCM in au/snd files?

I actually got the original Duck TM1 decoder compiled and hooked up to a small test bench app. It even runs without crashing. However, it produces unequivocally incorrect output:

I was expecting something more in a solid white for this particular frame. This doesn’t exhibit the normal Duck TM1 bidirectional artifacts that I’m used to seeing. I also wired up a number of different blitting calls with both 16- and 24-bit data, to no avail.

I’ll give the API calls another look to see if I’m missing anything obvious in that department (this approach is unlikely to bear fruit, considering the dearth of documentation). Failing that, I might find myself in the ironic position of using libavcodec’s native Duck TM1 decoder to verify the operation of the original decoder to see what is going wrong with files that are known to work correctly in lavc, so that the original decoder will work completely and yield clues about the large body of problem TM1 files.

Addendum: I decided to enlist Valgrind’s assistance against the adversary. This is the first time I have touched the utility in awhile. Let’s see what it has to say…

  valgrind: the 'impossible' happened:
     Killed by fatal signal

I’m not sure what to think about that. Should I feel proud? Fortunately, leading up to the impossible condition are plenty of other memory errors, all ripe for investigation.

It’s like digital archaeology– understanding the ancient (by internet time) workings of less advanced engineering efforts. Pop culture depictions of archaeology would have us believe that those who toil in the field are searching for that one artifact that would neatly solve the entire puzzle at hand. Historically, perhaps the artifact that came closest to fitting this archetype was the Rosetta Stone, and even that was incomplete (if I’m observing the pictures correctly).

So it is with my current investigation. Much like Sean Connery playing Indiana Jones’ dad searching for the Holy Grail in The Last Crusade, I run the risk of making the Duck TrueMotion 1 algorithm my lifelong obsession. I got that Duck TM1 source code into a state where I could compile it against a standalone program. It only required 38 C source files from the original vpvision source tree and a mess of attendant header files to boot.

Read the rest of this entry »

I don’t learn very quickly. A good illustration of this personal shortcoming is my ongoing battle with the Duck TrueMotion 1 video codec, a conflict that is well into its 8th year at this point. This was one of my first exposures to the field of multimedia hacking as I discovered some Duck TM1-encoded AVI files on select Sega Saturn console games all the way back in 1998. I found a few Japanese Windows programs that could play the files. Early in my multimedia hacking career, I worked hard to reverse engineer the TM1 algorithm.

In 2002, On2 (formerly Duck) open sourced their VP3 codec (which would later become Theora). I kept stubbornly trying to RE TM1 from the binary when Alex notified me that On2′s open source VpVision package contains the decoders for TrueMotion 1 and 2. It also included the decoders for Duck DK3 and DK4 ADPCM, both of which I successfully had RE’d from binary code, and at around the time that VpVision had been released as open source.

The VpVision source has been quite valuable in RE’ing both the TM1 and TM2 algorithms. It’s extremely difficult to understand, however, as is often the case with code produced on a deadline. Some TM1 files still fail to work correctly with the native FFmpeg decoder. I always thought it would be useful if I could compile the code and run it in a step debugger, or perhaps profile it with other tools, in order to sort out the correct operation when decoding certain files. But the code seemed like such a mess that I didn’t think it would compile on Linux. It looked like it would only compile on Windows or maybe Mac, and only with some effort.

Read the rest of this entry »

The FOURCC List affords us many scattered bits of intelligence on various codecs used throughout the history of multimedia technology. Duck/On2′s early TrueMotion codecs are assigned a variety of FOURCCs– such as DUCK, TMOT, and PVEZ– which may or may not refer to distinct TrueMotion variants.

Serge van den Boom informed me that the 3DO version of Star Control II made use of some TM variation named TrueMotion-S. The open sourced Star Control II effort includes code to decode this video. The relevant files are dukvid.[ch] in this directory. A sample (logo.tar.bz2) is available in my Duck TM1 samples directory. Interestingly, the bundle actually contains 4 files. The .duk file has all of the video data stuffed together. The .hdr file contains some header information. The .frm file contains the frame offset boundaries for the .duk data file. And the .tbl file apparently contains data for initializing the delta table to use for decoding the data.

I am not yet sure if the data is decoded to 16- or 24-bit video. If someone is willing to jump in and figure this out, it might help us sort out the remaining pieces of the generalized Duck TM1 decoder for FFmpeg. It stands to reason, however, that the data is 16-bit at best since Star Control 2 is such an early game in terms of the multimedia genre. The game is reportedly one of the earliest games that Duck TM1 was used for so it may be on the bottom rung of the Duck predictive evolutionary ladder.

Reimar Döffinger has submitted some work for FFmpeg’s Duck TrueMotion 1 decoder. Now the decoder can output a frame that looks reasonably correct but only paints the left half of the frame:

So we’re close, we’re very close on this variant. My guess would be that the OUTPUT_PIXEL_PAIR(), which is used in both the 16- and 24-bit variants, is not doing the right thing in the latter mode.

Further, Kostya has fixed some bugs in the TrueMotion 1 data tables that caused decoding problems with the data files from Phantasmagoria: A Puzzle of Flesh.

One more problem (which may or may not be a single problem): I still have several 16-bit TM1 samples that do not decode correctly. They decode as colorful static that looks like this:

Duck TM1 actually produces some very curious artifacts due to its bidirectional predictive coding method and this frame does not do the artifact bug justice. Incidentally, this particular frame comes from the the sampler disc for the Sega Saturn game Bug! and the file can be found in the samples archive: http://samples.mplayerhq.hu/V-codecs/DUCK/ (file is bugsampler-m01-16bit.avi).

What could be the problem? Many of the frames toss off the error message “help! truemotion1 decoder went out of bounds” which is a message I added in the early days of development to indicate that the decoder had exhausted the encoded bytestream. My first impulse is that there must be something wrong with the data tables. Seems unlikely, though, since I more or less ripped the tables directly from the original VpVision source. As for the tables I had to retype, Kostya has apparently corrected mistakes in those.

There are 3 large, important tables found in libavcodec/truemotion1data.h: pc_tbl2[], pc_tbl3[], and pc_tbl4[]. Where’s pc_tbl1[]? Darned if I know but I guess the original source didn’t need it or declare it. Tables #2 and 4 begin with 10 and 9 4-entry delta runs, respectively. Table #3, however, has no 4-entry delta runs. Thus, if table #3 were mistakenly selected instead of 2 or 4 and the bytestream had a good number of low-numbered delta indices, the bytestream would be exhausted sooner than it should be.

Where are these tables selected? In TM1′s overly-complicated header decoding logic. It may be time to review that for correctness.