Some of you may have noticed that the various websites hosted here on multimedia.cx were having a tad bit of difficulty recently. Long story short: My previous web host was having some serious problems and I decided it was time to ditch them and move on to a better one. Fortunately, I had (and continue to maintain) consistent, automated backups of everything hosted on multimedia.cx. But I really wasn’t looking forward to the task of finding a new provider. Whenever I have studied web host providers in the past, they all seem pretty much the same– offering UNLIMITED EVERYTHING!! along with perfect uptime and reliability for next to nothing^{(*** see details below in 5-point font)}. And most of their websites boast a design style reminiscent of the worst e-marketing sites and guaranteed to annoy the utilitarian, tech-savvy geek.

When it boils right down to it, I think I was being asked to make a decision regarding a new web host based on the female smiling at me on the front page. Honestly, these photos were generally the only distinguishing feature among the various services:

Miss 1&1 Hosting	Miss midPhase
Miss FastDomain

Read the rest of this entry »

I remember when the Mono people first announced the Moonlight project for Linux that would interoperate with Microsoft’s Silverlight. They claimed that Microsoft would release a special binary codec pack that would allow Linux users to play back their proprietary media codecs. However, this codec pack would not be allowed for use in any other application, like FFmpeg or GStreamer. How are they going to enforce that? Or so I wondered. Tonight I learned how.

I started investigating the API of the binary codec pack blobs a few weeks ago. I got as far as figuring out how Moonlight registers the codecs. Then I lost motivation, in no small part because there isn’t that much in the blob that I would deem interesting (perhaps one method for keeping people from sorting out the API). In the comments of the last post on the matter, people wondered if the codec pack included support for WMA Voice, which is still unknown. I can’t find any ‘voice’ strings in the blob. However, I do find references to lossless coding. This might pertain to Windows Lossless Audio, or it could just be a special coding mode for WMA3 Pro. Either way, I’m suddenly interested.

So I looked for interface points in the Moonlight source. Moonlight simply loads and invokes registration functions for WMA, WMV, and MP3. The registration functions don’t return any data that Moonlight stores. Moonlight doesn’t appear to load (via dlsym()) or invoke any other codec pack functions directly. So how can it possibly be interfacing? The only other way the interaction could flow is if the codec pack shared library was invoking functions in Moonlight…

Oh, no… they wouldn’t do that, would they?

Read the rest of this entry »

Thanks to Måns for modifying the FATE script in a way that supports automatically cross compiling FFmpeg for a different target CPU on a faster host machine, transferring the binary to a machine specimen that runs the target CPU in question, and remotely asking the target CPU to run the battery of FATE test specs. The upshot of all of this is that FATE is effectively running on an ARM-equipped Beagle Board and contributing results back that anyone can view via the main FATE page.

I hope to get his changes rolled into the main script soon. It’s great work, and I’m hard-pressed to name another continuous integration system that can operate on such diverse platforms, environments, and circumstances.

I dusted off my old Sega Dreamcast this evening — the one I used to do homebrew programming on — and enjoyed some games. As I was playing, I realized that the next evolution of FATE would be to get it to continuously run automatic cross-compile and test cycles on the Dreamcast’s SH-4 via a custom serial protocol, similar to what John Koleszar described in this comment.

But I have a few more FFmpeg code paths to cover before I can even think about that.

One weak spot in FATE‘s architecture has to do with encoding and muxing formats. So far, I have been content to allow the master regression suite handle the encode/mux tests for the most important formats that FFmpeg supports. But the suite doesn’t handle everything. Plus, I still have the goal of eventually breaking up all of the regression suite’s functionality into individual FATE test specs.

At first, the brainstorm was to encode things directly to stdout so that nothing ever really has to be written to disk. The biggest problem with this approach is that stdout is non-seekable. For formats that require seeking backwards, this is a non-starter (e.g., a QuickTime muxer will always have to seek back to the start of the file and fill in the total size of the mdat atom that was just laid down on disk).

So it’s clear that an encode/mux test needs to commit bytes to some seekable media. Where is it okay to write to disk? I think $BUILD_PATH should be okay, since the build step already writes data there.

The natural flow of the master regression suite is to encode/mux a test stream, run a hash of the resulting stream, then demux/decode the encoded stream and run a hash on the result. In FATE, I imagine these 2 operations would be split into 2 separate tests. But how can I guarantee that one will run before the other? Right now, there’s no official guarantee of the order in which the test specs run. But I can — and plan to — change the policy via fate-script.py so that the tests are always run in order of their database IDs. That way, I can always guarantee that the encode/mux test is run and the correct sample file is waiting on disk before the corresponding demux/decode test executes.

I also need a way to verify the contents of the encoded file on disk. I think this can be handled via a new special directive along the lines of:

{MD5FILE,$BUILD_PATH/output} $BUILD_PATH/ffmpeg -i $SAMPLES_PATH/input/input -f format -y $BUILD_PATH/output

This will read the bytes of the file ‘output’ and compute the MD5 hash. This seems simple enough in a local environment. But it is another item that may pose challenges in the cross-FATE architecture I am working on with Mans which will support automated testing on less powerful/differently-targeted platforms.

The FFmpeg team is still very much committed to making a formal release, and soon. Originally, the release was slated for this weekend. Some problems with the bug database made it difficult to host a major bug-fixing initiative as planned last weekend. So the current plan is to go on a binge bug-fix this weekend and hopefully release next weekend. The release has waited this long, so what’s one more week?

Meanwhile, things are going great with automated testing. Thanks to much discussion and determination from quite a few people, the entire regression suite passes, at long last, on more configurations than ever before, giving FATE a more solid baseline for continuous testing. Most notably, the regressions pass on 32- and 64-bit Mac OS X, 32-bit icc (Intel’s C compiler), and PowerPC/Linux when using gcc 4.0, 4.1, or 4.2. 4.3 still presents a problem, while the SVN versions of gcc for the PowerPC have been messed up for months. I’m really not sure what to do about that. Further, I see that gcc on PowerPC 64 suffers from a colorful variety of random problems (sometimes the compiler even comes up with an internal error, and we’re not even talking about the SVN versions of gcc here).

Still, things are looking up. Also, according to my tally on the FATE test coverage page, FFmpeg supports 501 muxers, demuxers, encoders, and decoders. I don’t have to tell you that nothing else comes close.

I was visiting ossguy’s blog today when I noticed that he took a small break from the usual “Free Software Über Alles” rhetoric to post a useful investigation of the Microsoft binary codec pack that corresponds to Moonlight, Linux’s free implementation of Silverlight. At first, I was surprised to hear that this codec pack was finally available– I didn’t think it was going to be generally available until Moonlight’s official release. A little digging revealed that Moonlight 1.0 was officially released yesterday. I wondered why I hadn’t seen anything about this on any major Linux news sites yet.

Apparently, no one cares.

Well, I care, insofar as this is another way to study some codecs. I think it’s really slick that the codec pack is one monolithic, relatively small, binary blob that contains all the proprietary codecs needed to support Silverlight. ossguy’s post details 2 more-or-less direct download links:

• x86_32 version: silverlight-media-pack-linux-x86-5-1.so @ http://go.microsoft.com/fwlink/?LinkId=133186
• x86_64 version: silverlight-media-pack-linux-x64-5-1.so @ http://go.microsoft.com/fwlink/?LinkId=133816

I see that compn is already on the case, identifying the precise codec formats that this blob is designed to handle: wma1, wma2, wma3, wmv1, wmv2, wmv3, wmav, wvc1, and mp3. So, really, nothing interesting for our cause. Almost all of those formats are already supported in FFmpeg. The one that isn’t — WMA3 — is in progress via a Summer of Code project. Who knows? Perhaps this codec pack could yield some new intelligence. But I tend to think the previous binary decoder released for Linux — packaged with Linspire — was pretty thorough in its presentation of symbols.

This codec pack is pretty thorough in the symbols department as well. Run ‘strings’ against the blob to see the ASCII strings. Filter the output through ‘c++filt’ as this will demangle (official technical jargon) the C++-style names:

  strings file.so | c++filt

And if you want to disassemble the binary, here is a little something I wrote up regarding the bare essentials of ‘objdump’ as applied to reverse engineering work.

At the very least, I can see this codec pack being useful for hooking up to FFmpeg in order to gather a baseline for profiling to see how FFmpeg’s decoders stack up. Further, I should be able to use it to decode reference samples to verify how close FFmpeg decodes, e.g., WMA 1/2 data to the original.

Whatever the case, I have started a MultimediaWiki page to describe the API.

Remember in my last post when I described that the reference encodings in the MPEG-1 audio conformance suite were stored as a list of 32-bit hex numbers in ASCII format? I just thought I would mention that that was only for the layer I encodings. The layer II encodings, for whatever reason, only have 1 byte per line in ASCII format. The layer III encodings are in a proper binary form, however.

Anyway…

Now that I am confident that the root mean square (RMS) tests pass, I need to decide how to store the samples and in which numerical precision and format the RMS will be computed. At first I reasoned that, since the 24-bit integer, 32-bit integer, and 32-bit float precisions all yielded passing results, any should work. However, before I got enough precision in the FFmpeg output, the 32-bit float precision failed where the other 2 still succeeded. This leads me to believe that the 32-bit float space would be the best precision to work with.

However, some tests reveal that either I’m doing something wrong, or FFmpeg has a bug in which it flips sign on individual samples when converting to a floating point format. My money is on the former (i.e., my mistake). However, I then realized that there is really no reason to ask FFmpeg to output floating point data from its various MPEG-1 audio decoders since they are all decoding to integers anyway. However, I do need to perform some configuration rework in order to compile FFmpeg in such a way that it will output 32-bit precision integers via configuration option vs. manual hacking.

So my proposed testing process for the MPEG-1 audio conformance vectors is the following:

• patch FFmpeg to allow for a –enable-audio-long configure option that will allow audio decoders to output higher precision audio (only applies to MPEG-1 decoders right now)
• convert all of the encoded samples to proper binary files
• convert all of the conformance vectors to s32le raw format; while this is 33% more data than is strictly necessary, I think it will be easier to process chunks of 32-bit data vs. 24-bit
• stage the encoded samples and reference waves in the formal FATE suite
• modify fate-script.py to honor a new command in the form of {RMS,$SAMPLES_PATH/wave-n-ref.s32le,37837.0} $BUILD_PATH/ffmpeg -i $SAMPLES_PATH/wave-n.mpg -f s32le -; the first parameter of the RMS special directive is the file of raw, 32-bit, signed, little endian data against which the command output must be compared, while the second parameter is the RMS threshold not to be exceeded (in this case, 2^32-15 / sqrt(12) = 37837, see last entry for explanation)
• enter new FATE test specs

Now that I write it all out, however, I realize that it is not strictly necessary to get FFmpeg modified to output higher precision numbers since the 16-bit numbers, scaled up, will pass the 24- or 32-bit thresholds, per my empirical findings. This makes me wonder if I should store and read the data as 32-bit integers (and enable high precision from FFmpeg), but then convert the numbers to floating point for the RMS calculation. The performance impact would be negligible (getting all the numbers lined up in arrays still takes longer than doing floating point ops on them), and the test would be stricter and conceivably catch more problems. Then again, it may have been a math error on my part that caused the floating point test to fail while the 24- and 32-bit tests worked.

One more stipulation I (may) need to make in the final test: The reference wave always has considerably more samples (e.g., 65536) than FFmpeg decodes (e.g., 37632). I have been performing RMS along the length of the shorter wave and the test has been meeting threshold. I still don’t know if this is a discrepancy to worry about, but at the very least, I think I should add a provision in the ad-hoc {RMS} method that the decoded wave has to be at least half as long as the reference wave.

You would not believe how many numerical gymnastics I have to perform in order to test these MPEG-1 audio conformance vectors. It seems straightforward enough– a conformance vector, at least for layers 1 and 2, consists of a .MPG file and a .PCM file. The MPG file is supposed to contain an encoded MPEG audio stream while the PCM file has the output after the corresponding MPG file has been run through the official reference decoder. The root mean square (RMS) of the difference between that reference PCM file and, say, the output of the FFmpeg decoder needs to be less than 1 / (32768 * sqrt(12)). So what’s the big deal?

Read the rest of this entry »

I have a specific task I need to do: I need to view a DV stream captured from an NTSC video source via DV bridge and capture screenshots. Since there is a lot of footage, I need to be able to seek easily through the stream (not unreasonable since DV is purely intra-coded, i.e., it consists of independently coded keyframes). Since this is television source material, I want the data competently deinterlaced while viewing and certainly when the screenshots are taken. One more thing — I want the screenshots to retain their 720×480 resolution that looks correct on a computer monitor, and not be squished down to 640×480, overscan region and all.

If you must know, it’s because I play lots of games for my Gaming Pathology project and contribute data for them — including screenshots — to the MobyGames database for posterity. I have a huge number of console games yet to cover and the screenshot dilemma is one thing that is holding me up. Consider the need to properly preserve representative screenshots of the Sega Saturn version of Space Jam for future generations of video game historians to study.

I didn’t think this would be such a tall order. Read the rest of this entry »

I lost the thread on those pre-release Gdium netbooks sometime ago. After being accepted into the One Laptop Per Hacker program, I responded affirmatively but never heard anything else. So how did Måns Rullgård manage to get ahold of one?

It’s all good. Check out the link to his blog for all the surface details of the OS and default software. The important thing is that we have a computer with a MIPS 64-bit CPU in the FFmpeg developer community. It should only be a matter of time before the unit starts serving FATE duty alongside the Alpha in Måns’ flat.