I remember when the Mono people first announced the Moonlight project for Linux that would interoperate with Microsoft’s Silverlight. They claimed that Microsoft would release a special binary codec pack that would allow Linux users to play back their proprietary media codecs. However, this codec pack would not be allowed for use in any other application, like FFmpeg or GStreamer. How are they going to enforce that? Or so I wondered. Tonight I learned how.

I started investigating the API of the binary codec pack blobs a few weeks ago. I got as far as figuring out how Moonlight registers the codecs. Then I lost motivation, in no small part because there isn’t that much in the blob that I would deem interesting (perhaps one method for keeping people from sorting out the API). In the comments of the last post on the matter, people wondered if the codec pack included support for WMA Voice, which is still unknown. I can’t find any ‘voice’ strings in the blob. However, I do find references to lossless coding. This might pertain to Windows Lossless Audio, or it could just be a special coding mode for WMA3 Pro. Either way, I’m suddenly interested.

So I looked for interface points in the Moonlight source. Moonlight simply loads and invokes registration functions for WMA, WMV, and MP3. The registration functions don’t return any data that Moonlight stores. Moonlight doesn’t appear to load (via dlsym()) or invoke any other codec pack functions directly. So how can it possibly be interfacing? The only other way the interaction could flow is if the codec pack shared library was invoking functions in Moonlight…

Oh, no… they wouldn’t do that, would they?

Read the rest of this entry »

Rejoice! Thanks to the inimitable multimedia hacker Allan Smithee, FFmpeg now supports the XIntra8 coding scheme! Why is this important? Also known informally as the J-frame (and X8Intra and IntraX8), the XIntra8 coding mode has long been the missing piece of the Microsoft WMV2 puzzle and is apparently also used in certain variations of WMV3.

Update: Check out Kostya’s rant on the matter for better details on exactly what XIntra8 is, and why it is so painful.

This has been a longstanding problem for FFmpeg’s open source WMV2 decoder. You would be watching a WMV2-encoded video and suddenly, obnoxiously, there would be a severe glitch where you could watch blocks of incorrect colors moving around the screen. For example:

The XIntra8 is a different type of intraframe from the usual I-frame found in WMV2. Since the decoder could not handle the data, the policy was to just copy over the previous interframe and proceed with more frames, and hope that a regular I-frame was not too far in the future.

But now, I can finally properly watch this WMV2 encode of Britney Spears old music video for “Not A Girl, Not Yet A Woman”:

Oh, don’t try to claim that you don’t have an extensive collection of her works. It’s okay to state that you have amassed the collection strictly for the academic purpose of multimedia study. That’s my story and I’m sticking to it.

Pursuant to DVD Jon’s post about compiling and using the SMPTE reference VC-1/VC-9/WMV3/WMV9 decoder with VLC, I wrote an FFmpeg module to use the same reference code. Next, I will try to adapt it to xine. Though I hope we do not grow too collectively complacent with this decoder.

I tried decoding 3 different WMV3 videos from 3 unique sources. The reference decoder seemed to work fine on 2 videos but not on the third.

The third videos fails on all frame decodes, usually tripping errors about “FATAL ERROR: DecodeACRunLevel: AC coef array overrun”.

Thanks for Benjamin Larsson for advising me on how to make a proper graph using Graphviz (specify a graph of type ‘prof’ vs. ‘unix’). Thus, here are some nice (and big) graphs based on the call/ret experiment at this point:

• wmv2init() callgraph
• wmv2packet() frame #0 decode callgraph
• wmv2packet() frame #18 decode callgraph

See this post (“Refining The call/ret Monitor”) for details on how the graph data is generated.

And as a bonus, I put the toolset to work analyzing Linspire’s WMV3/WMV9 decoding module. Here are some call graphs (I used the teaser trailer from Halo 2, hence the “halo2″ in the filenames):

• wmv3init() callgraph; I like the way the graph expresses countless functions invoking wmvMalloc() which simply calls malloc()
• wmv3packet() frame #0 decode callgraph; note that this is just a plain, black I-frame and does not exercise a lot of the logic in the module; let’s start small

Reverse engineering is, of course, a tedious, time-consuming, and error-prone task. It requires a lot of concentration that I either do not have or do not care to invest in the RE task. That is why so many of my RE experiments are geared toward automating the task as much as possible. To that end, I am optimistic about this call/ret monitoring experiment since it yields such a good high-level overview of an algorithm contained in a binary (with debug symbols). But it can use some improvement(s):

Read the rest of this entry »

I have been at it again, concocting more highly specialized reverse engineering experiments. If you have been reading my blog for awhile and are familiar with my methods, or lack thereof, you know I like to try random stuff in the hope that I may accidently come across a good idea.

So we have these WinCE binary modules with debug symbols that implement various advanced Microsoft media codecs. Then we also have the Microsoft media modules that Linspire licensed (provided you know where to find them) that also have copious debug symbols. I wanted to put this intelligence to good use.

Read the rest of this entry »

I updated my main multimedia.cx site tonight. First, I finally added a link to the formal Nullsoft Video (NSV) specs that Nullsoft/AOL saw fit to release. I am ecstatic to not have to worry about maintaining my NSV format document any longer.

Next, I have carefully documented the binary API that Linspire’s version of FFmpeg uses to interface to the licensed WMV2, WMV3, and WMA3 binary modules distributed with Linspire’s package. Who knows? Such knowledge could come in handy someday.

One of Linspire’s big selling points is that it supports Microsoft Windows Media decoding out of the box. How does it do this? Several colleagues have written and informed me that Linspire has licensed binary decoding modules from Microsoft. Linspire’s packaged distribution comes with such x86/Linux-native shared objects as libwma2.so, libwma3.so, libwmv2.so, libwmv3.so.

So why is this such a big deal? My informants tell me that the binaries are un-strip’d which means that they have a lot of debugging symbols packaged inside. Thanks, Linspire.

I’m feeling good about these new MS multimedia libraries with debug symbols. At first I was a little disappointed to see that ‘only’ WMA8, WMA9 and WMV9 were covered by these libraries.

Read the rest of this entry »

Since Microsoft was kind enough to freely release certain of their multimedia libraries with extensive symbolic information, I thought I would go one step further and start extracting useful intelligence from the libraries in a methodical manner. To that end, I wrote a Perl script that analyzes the disassembly output of a “dumpbin.exe /disasm <file>” command (standard with the Microsoft development tools) and outputs a list of all the functions in the file as well as all the functions the function calls. The list is divided into functions that are not called within the code (these are suspected to be top-level functions) and functions that are referenced.

Read the rest of this entry »