I remember when the Mono people first announced the Moonlight project for Linux that would interoperate with Microsoft’s Silverlight. They claimed that Microsoft would release a special binary codec pack that would allow Linux users to play back their proprietary media codecs. However, this codec pack would not be allowed for use in any other application, like FFmpeg or GStreamer. How are they going to enforce that? Or so I wondered. Tonight I learned how.

I started investigating the API of the binary codec pack blobs a few weeks ago. I got as far as figuring out how Moonlight registers the codecs. Then I lost motivation, in no small part because there isn’t that much in the blob that I would deem interesting (perhaps one method for keeping people from sorting out the API). In the comments of the last post on the matter, people wondered if the codec pack included support for WMA Voice, which is still unknown. I can’t find any ‘voice’ strings in the blob. However, I do find references to lossless coding. This might pertain to Windows Lossless Audio, or it could just be a special coding mode for WMA3 Pro. Either way, I’m suddenly interested.

So I looked for interface points in the Moonlight source. Moonlight simply loads and invokes registration functions for WMA, WMV, and MP3. The registration functions don’t return any data that Moonlight stores. Moonlight doesn’t appear to load (via dlsym()) or invoke any other codec pack functions directly. So how can it possibly be interfacing? The only other way the interaction could flow is if the codec pack shared library was invoking functions in Moonlight…

Oh, no… they wouldn’t do that, would they?

Read the rest of this entry »

Rejoice! Thanks to the inimitable multimedia hacker Allan Smithee, FFmpeg now supports the XIntra8 coding scheme! Why is this important? Also known informally as the J-frame (and X8Intra and IntraX8), the XIntra8 coding mode has long been the missing piece of the Microsoft WMV2 puzzle and is apparently also used in certain variations of WMV3.

Update: Check out Kostya’s rant on the matter for better details on exactly what XIntra8 is, and why it is so painful.

This has been a longstanding problem for FFmpeg’s open source WMV2 decoder. You would be watching a WMV2-encoded video and suddenly, obnoxiously, there would be a severe glitch where you could watch blocks of incorrect colors moving around the screen. For example:

The XIntra8 is a different type of intraframe from the usual I-frame found in WMV2. Since the decoder could not handle the data, the policy was to just copy over the previous interframe and proceed with more frames, and hope that a regular I-frame was not too far in the future.

But now, I can finally properly watch this WMV2 encode of Britney Spears old music video for “Not A Girl, Not Yet A Woman”:

Oh, don’t try to claim that you don’t have an extensive collection of her works. It’s okay to state that you have amassed the collection strictly for the academic purpose of multimedia study. That’s my story and I’m sticking to it.

Pursuant to DVD Jon’s post about compiling and using the SMPTE reference VC-1/VC-9/WMV3/WMV9 decoder with VLC, I wrote an FFmpeg module to use the same reference code. Next, I will try to adapt it to xine. Though I hope we do not grow too collectively complacent with this decoder.

I tried decoding 3 different WMV3 videos from 3 unique sources. The reference decoder seemed to work fine on 2 videos but not on the third.

The third videos fails on all frame decodes, usually tripping errors about “FATAL ERROR: DecodeACRunLevel: AC coef array overrun”.

Thanks for Benjamin Larsson for advising me on how to make a proper graph using Graphviz (specify a graph of type ‘prof’ vs. ‘unix’). Thus, here are some nice (and big) graphs based on the call/ret experiment at this point:

• wmv2init() callgraph
• wmv2packet() frame #0 decode callgraph
• wmv2packet() frame #18 decode callgraph

See this post (“Refining The call/ret Monitor”) for details on how the graph data is generated.

And as a bonus, I put the toolset to work analyzing Linspire’s WMV3/WMV9 decoding module. Here are some call graphs (I used the teaser trailer from Halo 2, hence the “halo2” in the filenames):

• wmv3init() callgraph; I like the way the graph expresses countless functions invoking wmvMalloc() which simply calls malloc()
• wmv3packet() frame #0 decode callgraph; note that this is just a plain, black I-frame and does not exercise a lot of the logic in the module; let’s start small