Reimar Döffinger has submitted some work for FFmpeg’s Duck TrueMotion 1 decoder. Now the decoder can output a frame that looks reasonably correct but only paints the left half of the frame:

So we’re close, we’re very close on this variant. My guess would be that the OUTPUT_PIXEL_PAIR(), which is used in both the 16- and 24-bit variants, is not doing the right thing in the latter mode.

Further, Kostya has fixed some bugs in the TrueMotion 1 data tables that caused decoding problems with the data files from Phantasmagoria: A Puzzle of Flesh.

One more problem (which may or may not be a single problem): I still have several 16-bit TM1 samples that do not decode correctly. They decode as colorful static that looks like this:

Duck TM1 actually produces some very curious artifacts due to its bidirectional predictive coding method and this frame does not do the artifact bug justice. Incidentally, this particular frame comes from the the sampler disc for the Sega Saturn game Bug! and the file can be found in the samples archive: http://samples.mplayerhq.hu/V-codecs/DUCK/ (file is bugsampler-m01-16bit.avi).

What could be the problem? Many of the frames toss off the error message “help! truemotion1 decoder went out of bounds” which is a message I added in the early days of development to indicate that the decoder had exhausted the encoded bytestream. My first impulse is that there must be something wrong with the data tables. Seems unlikely, though, since I more or less ripped the tables directly from the original VpVision source. As for the tables I had to retype, Kostya has apparently corrected mistakes in those.

There are 3 large, important tables found in libavcodec/truemotion1data.h: pc_tbl2[], pc_tbl3[], and pc_tbl4[]. Where’s pc_tbl1[]? Darned if I know but I guess the original source didn’t need it or declare it. Tables #2 and 4 begin with 10 and 9 4-entry delta runs, respectively. Table #3, however, has no 4-entry delta runs. Thus, if table #3 were mistakenly selected instead of 2 or 4 and the bytestream had a good number of low-numbered delta indices, the bytestream would be exhausted sooner than it should be.

Where are these tables selected? In TM1′s overly-complicated header decoding logic. It may be time to review that for correctness.

Some time ago, Alex Beregszaszi and I created an FFmpeg video decoder to handle Duck TrueMotion 1 data. However, there are two major variations of this data format: 16-bit and 24-bit. The FFmpeg decoder presently only handles 16-bit data. There is a non-negligible number of games from the mid- to late-1990s that used this format for their FMV and many of them use the 24-bit variant.

Virtua Cop 2, Sega Saturn,
one of the Duck TM1-using games that uses the 16-bit variant

Read the rest of this entry »

Kostya has ironed out the details of the Duck TrueMotion 2 video codec format and finished the FFmpeg decoder. Now it is possible to natively decode all 100+ AVI files on the PC version of Final Fantasy VII:

Check out the FFmpeg CVS repository.

Kostya has plodded through the long-open sourced, yet highly cryptic, code of On2′s VpVision and successfully reverse engineered a description of the Duck TrueMotion 2 video codec (fourcc: TM20). He then re-implemented a fresh version which is available in the FFmpeg CVS repository. Here it is in action, decoding the one known TM20 sample:

The video depicts something related to the unkillable Final Fantasy game series. The most famous application for the TM20 codec has been to encode the FMV in Eidos’ PC port of Final Fantasy VII. This game is known to use a bitstream-incompatible version of the TM20 format. So there is still some work to do on this codec before the Final Fantasy fanboys rejoice over being able to natively decode the FMV for this game.

Also, do you remember the basic Autodesk FLIC format? Did you know there are more variations of the format than you can possibly imagine? As much as I enjoy hacking on esoteric multimedia formats I just could not bring myself to care about these variations. This is mostly because I did not have sample files.

Recently, however, Steven Johnson has seen fit to implement several variations of the format including FLX and DTA extensions. The code is in FFmpeg’s CVS.

Do you have any idea how long I have been involved in multimedia hacking and reverse engineering? About 5 years now. All that while, folks have warned me sternly, and constantly, that this type of work would get me sued to death. I am pleased to announce that today I received my first legal threat. I feel that my work has finally been validated! Well, it was not necessarily a legal threat, like those notorious “nastygram” cease & desist letters. It was more like a veiled reference to a possible future legal threat. Someone identifying himself as the assistant general counsel for On2 said that the company took exception to the fact that I was posting decompilations of their Java decoder. And just when I was starting to feel that no one cared about my work…

Naturally, this raises some pressing questions. First and foremost, why was I contacted by the assistant general counsel? Why doesn’t my case warrant the attention of the lead/primary/head general counsel? Maybe if I went after their latest generation codec, VP7, my actions would merit an escalation.

For the time being, I have decided to not post the Java decompilations on my Practical Reverse Engineering site. This entire site is partially an experiment to test where the limits are. Looks like we found one such limit.

I never had a compelling reason to research legal options surrounding these RE activities. Maybe it is time to start. But I am just so lazy… As always, this subject may be revisited. Feel free to email me regarding this situation.

I am pleased to report that people have been jumping on the decompiled Java-based VP5 decoder. Notably, it was clear to one individual that the method called clinch() that read bits from the stream using some bizarre algorithm is, in fact, an arithmetic decoding algorithm.

Regarding credits: I realize that many people engaged in reverse engineering activities are a little paranoid about having their achievements recognized. As such, my default policy is to not mention a contributor’s name unless they specifically ask for credit.

This is the current version of the decompiled VP5 Java cryptogram. Updates may occur at any time so check on the version number and timestamp inserted by SVN.

I don’t know why this did not occur to me sooner: Distributed reverse engineering!

Read the rest of this entry »

So I have managed to automatically de-obfuscate an obfuscated Java project. Remember, there are 2 major challenges in reverse engineering: 1) Understanding the original code flow, and 2) understanding what the original identifier names could have been. My experiment was focused on problem #2. Problem #1 is generally a non-issue in decompiled Java code since Java classes retain so much information about the original code flow.

Are there better approaches for obfuscating Java code?

Read the rest of this entry »

And software companies wonder why we users have trouble taking software end-user license agreements seriously. Roberto Togni actually read the license that accompanies On2′s VP7 Decoder. It contains this clause:

You may NOT:
4.publish or provide any results of tests, including without limitation benchmark tests, run on the Software to any third party without On2′s prior written consent

In fairness to On2, this is not an uncommon clause in EULAs. However, it presents some very curious scenarios. Am I allowed to publish something like this?

I looked at a VP7 sample and it did not look as good as WMV9, H.264, or even On2′s own VP6.

Maybe I should write and ask permission. A colleague brought up another point: Since this “no benchmark tests” is such a common EULA clause, it should stand to reason that Microsoft’s Windows Media Encoder carries the same license. It is extremely unlikely that Microsoft would have granted written permission for this whitepaper exercise.

Update: Again, to be fair, the decoder license (On2′s Truecast Player) does not appear to mention anything about publishing benchmark tests. That is on the limited trial codec license.

I was just thinking the other day about whether On2 would release VP7. This is the organization that brought us TrueMotion 1, TrueMotion 2, VP 3.0, VP 3.1, VP 4.0, VP 5.0, VP 6.0, and VP 6.1, not to mention VP 6.2. VP70 just seems like a natural progression.

What does ‘VP’ stand for anyway? I have never read any definitive answer. However, if On2′s technology focus is any clue, it stands for “Video Predictor”, owing to On2′s almost religious devotion to prediction-based compression algorithms.

Michael Roitzsch tipped me off that On2 has, indeed, unleashed their VP7 codec upon the world. True to form, their literature asserts that VP7 “rUleZ!!1!” and that “MicRo$$oFt is teh suck!1!!!” (slightly paraphrased, they state their points using graphs).

Read the rest of this entry »