Breaking Eggs And Making Omelettes

Topics On Multimedia Technology and Reverse Engineering


Meta:

Granular Regression Tests

August 25th, 2008 by Multimedia Mike

I have been studying the FFmpeg regressions test infrastructure ever so carefully so I can decide the best way to convert all of its functionality and test coverage into the FATE system as individual test specs. I have some ideas but I never know when I will have both the time and the motivation to implement them. In the interim, I am retiring the master ‘make test’ full regression test spec. In studying the regression architecture, it’s immediately obvious that ‘make test’ invokes these 3 sub-regression suites, which each have their own test specs now:

That should help until I break down the tests even further. Oh, and I also added a test spec for ‘ffmpeg -h’. Hey, FFmpeg got into a state once where it segfaulted on help. We don’t want to let that go uncaught again.

Posted in FATE Server | 5 Comments »

GSoC Showcase: ALAC

August 24th, 2008 by Multimedia Mike

For his FFmpeg Summer of Code project, Jai Menon successfully completed an encoder for the Apple Lossless Audio Codec (ALAC). As you can see, Jai’s encoder is competitive with Apple’s iTunes, as well as dBpoweramp, another program that implements an ALAC encoder:


ALAC compression comparison chart

Thanks to Benjamin Larsson for setting up the graph. We’re rooting for the cyan bar to be the lowest one of each bar quartet. The code is already in the mainline tree.

The 13 samples I used for this test come from the OpenMusic CD and showcase a reasonable diversity of genres. The raw rips are available here.

Posted in Open Source Multimedia | 2 Comments »

Poor Amarok?

August 23rd, 2008 by Multimedia Mike

The MOOBEX project was really the first real exposure I had to the Amarok application. For the uninitiated, Amarok is a rather full-featured KDE-based music playing application and has been around since 2003. Back when I used Linux more exclusively, I stuck to xine directly for all of my media playback needs.

How was my first impression of Amarok? Well… it’s hard to say. The program was different on each distribution that I tried. In OpenSUSE, I was thoroughly frustrated when I first tried to play a file through Amarok, or rather, when I selected an audio file through the file manager and just let the system do its thing. Amarok popped up and just sort of stared at me. Okay, I recognize that I’m throwing a bunch of formats toward Amarok that the distro has administratively determined to be “legally unsafe”. But I should still get some obvious feedback to that effect. Oh, wait, look way down there in the status bar:


OpenSUSE Amarok error message

When I launched an audio file on another KDE-based distro installation (Mandriva), I expected the same behavior from Amarok. To my surprise, the program threw a far more brazen error. Ubuntu has Amarok version 1.4.9.1. Mandriva sports 1.4.8. The Amarok version in Mandriva exhibited the superior UI characteristic. I understand the source of the unsupported codecs. But I wonder about the dialog discrepancy. I am not sure if this dialog behavior is a version specific thing in Amarok, or a configuration option in Amarok, or an dialog characterstic configurable in the hosting KDE libraries. It’s a minor matter, to be sure, but it still makes me wonder how individual distributions would be able to devalue something like the Adobe Flash Player if it were open source and allowed distros to make “choice” changes to the functionality.

I’m sure that the Amarok developers worked very hard to produce the best music player they could. And it probably hurts to see their fruits of their labor mangled by various distributions to the point of practical worthlessness. I know the feeling based on my work on xine.

But here’s a huge annoyance with Amarok which I suspect is a core behavior: stop should mean STOP! When I did get a file to play and then tried to stop playback, the audio kept right on going long enough to make me think that something was wrong. Eventually, the audio started to gently fade out. This is not reasonable behavior. The expected action for a stop button is an immediate halt.

Posted in Open Source Multimedia | 1 Comment »

AAC Decoder Is In!

August 22nd, 2008 by Multimedia Mike

It certainly has been a long journey for native Advanced Audio Coding (AAC) in FFmpeg. It started with a Google Summer of Code project back in FFmpeg’s inaugural FFmpeg SoC season (2006). It went unfinished. Since then, many people have endeavored to fix it up to the point where it can be included into the mainline. But it was Robert Swain who persevered toward the end goal. And now look:

$ ffmpeg -formats
[...]
Codecs:
 D V    4xm             4X Movie
 D V D  8bps            QuickTime 8BPS video
 D A    8svx_exp        8SVX exponential
 D A    8svx_fib        8SVX fibonacci
 D A    aac             Advanced Audio Coding
[...]

Robert profiled the new AAC decoder to be significantly faster than the libfaad, the prevailing AAC decoding solution in the open source community. Further optimization work is ongoing, as is support for more advanced coding modes. Currently, the decoder only deals with low complexity (AAC-LC), the most common variant you are likely to encounter.

And of course, thanks also to Robert for creating more FATE work for me. I can’t avoid the problem of testing perceptual audio decoders for much longer.

Posted in FATE Server, Open Source Multimedia | 5 Comments »

Baldur’s Task

August 21st, 2008 by Multimedia Mike

I have a lot of experience consuming multimedia on Linux using open source software. But I have little experience generating or transcoding multimedia within the same parameters. So, much like I experienced with the MOOBEX exercise, I decided to move out of my comfort zone and work on transcoding for once.

Fortunately, I have a concrete task– a colleague who is interested in game preservation wanted to know about transcoding the Interplay MVE files from games such as Baldur’s Gate into a format a little more modern. So let’s see what we can do about using FFmpeg to transcode to MP4/H.264/AAC.


Baldur's Gate logo

I suspect the steps will look like this:

  • Download and install libfaac, as Kostya’s AAC encoder is not in the mainline tree yet
  • Download and install a recent snapshot of x264
  • Update to the latest SVN snapshot of FFmpeg and compile it with libfaac and x264 support
  • Transcode away, preferably with some manner of batch transcode process

Let’s see if it’s that easy.

This is the first time I have ever worked with x264. I decided to install from source. It has trouble finding and using the yasm assembler installed on the system. No matter, I’ll be patient for the test encodings. I finally got everything compiled together and figured out what options FFmpeg wanted (‘-acodec libfaac -vcodec libx264′) and I let it rip…

…and the whole thing blows up. Segfault down in the x264 library.

Between this and the MOOBEX exercise and the library problems on my little work project, I can’t help but take a dim view of multimedia support on Linux this week. Which is especially depressing since I sort of consider multimedia on Linux to be my beat.

Dog food sucks. But on the plus side, Robert finished committing AAC-LC decoding support to FFmpeg.

Posted in Open Source Multimedia | 4 Comments »

YASM Active

August 20th, 2008 by Multimedia Mike

Thanks to Loren Merritt for restructuring FFmpeg’s build system to use YASM and for submitting and relicensing a number of ASM optimizations compilable with YASM. The idea is that if you have YASM installed (x86_32 or x86_64), FFmpeg’s configure script will notice it and automatically compile in the new optimizations. I just installed the assembler on both the x86_32 and x86_64 FATE build machines.

Hope it works, and that it’s faster than before. I look forward to assessing how it improves performance on certain H.264 conformance vectors, particularly monsters like MV1_BRCM_D. From the associated README file:

“Check that the decoder handles the worse case of prediction bandwidth. Prediction bandwidth is at maximum due to largest number of motion vectors (in 1/4 pel position) per macroblock pair (32 as defined in standard). Non-integer position motion vectors require using 6-tap filter always.”

I’m not sure what all of that means. I just know that it takes a long time to decode on the FATE machines.

Posted in FATE Server | No Comments »

Introducing MOOBEX

August 19th, 2008 by Multimedia Mike

I was sitting at The Linux Foundation’s Collaboration Summit last April in Austin, Texas, USA pondering the state of multimedia usability on Linux. I realized that I live a somewhat sheltered computing existence vis-à-vis consuming multimedia because I happen to know a little bit more about it than most users. Then I got to wondering what the typical uninitiated user experiences when trying to play multimedia in Linux these days.

That’s the moment when I conceived MOOBEX: Multimedia Out-Of-Box EXperience. (Initially, I called it MOOB. But I wisely googled to see if that already had some meaning attached to it. I’m glad I did, and I’ll leave it to the reader to learn about the colloquial connotations of that word if they feel so inclined.)

For MOOBEX, I raided my vast personal multimedia archive and assembled a small suite of what I consider to be representative multimedia types. Then I developed a strict process for testing the multimedia capabilities of various stock Linux distributions from the perspective of a new Linux user. I tested the latest versions of 5 Linux distributions for the first pass at this. These include (x86_32 in all cases):

Fairness (and general curiosity) dictates that I give equal treatment to non-Linux operating systems. Thus, this initial pass also covers:

Executive Summary: Mandriva is absolutely remarkable with its out-of-the-box multimedia capability. Ubuntu is almost as good– while it isn’t too capable out of the box, it does a fine job of holding the user’s hand while installing required multimedia software on demand. The custom Xandros install on the Eee PC is also very good, albeit with a few holes. OpenSUSE isn’t too capable. Fedora Core’s multimedia support is deplorable and stultifyingly patronizing to boot (but, to be fair, somewhat improved from 2 years ago).

Apple’s and Microsoft’s operating systems perform as expected, able to play their respective company’s technologies perfectly out of the box but not being able to deviate from that, save for Adobe Flash.

I will have more analyses and opinions about my findings soon. In the meantime, the raw data is available in the MultimediaWiki. Also, be advised: No one can play an AIFF file anymore (except Apple). Maybe I’m the only person who finds this surprising; AIFF files weren’t that uncommon, were they?

Just as I was finishing this first round of reports, I realized that I named one sample girls-just-wanna-cook.rm. Wow, that could get me in trouble. I took the FUN_RM_32.rm sample from the samples archive — a 15-second sample of Cyndi Lauper’s timeless “Girls Just Wanna Have Fun” and renamed it to denote that it uses the RealAudio Cooker ‘COOK’ codec. Oops. Well, I’m not changing all those reports now.

Posted in Open Source Multimedia | 1 Comment »

The Intersection of Code And Money

August 18th, 2008 by Multimedia Mike

Some colleagues have advised me to lend my commentary regarding Jeff Atwood’s much-discussed piece entitled “Is Money Useless to Open Source Projects?” (or, as the Linux Hater put it, “You can’t even pay them to not suck“). I think I was supposed to argue the position that FFmpeg provides an efficient mechanism to convert raw currency into high quality multimedia source code. But I’m not sure I make that case or refute Jeff’s piece; in fact, I largely agree with his post. However, I can corroborate that FFmpeg does sometimes benefit from raw currency.

Let’s talk about the most visible example of the cash->code thesis as it relates to FFmpeg: Google’s Summer of Code sponsorship of FFmpeg. In 3 years, FFmpeg has earned 22 combined project slots; each slot represents a transfer for US$5000 from Google -> people involved in the project. Wow, this is the first time I have realized that Google has, to date, committed over 100 G’s to our silly little multimedia project. What has it gotten for that investment? Well, it got the foundation that made YouTube possible. What has FFmpeg been able to do with all that green? Out of 13 projects in 2006 and 2007, only 4 have currently reached acceptable status and are part of the mainline code tree.

Fortunately, this year, we have imposed much more stringent standards for the projects; students must complete their features to a level that they can be incorporated into the main tree by the end of the project. This week marks the end of GSoC 2008, so we will see how that turns out. If you have been following the FFmpeg mailing lists, you no doubt noticed a flurry of activity as students race to get all parts of their various projects approved for inclusion into the mainline tree.

Apart from GSoC, there are tales of different parts of FFmpeg development being actively funded by various organizations. The file dnxhdenc.c carries the following header:

 * VC3/DNxHD encoder
 * Copyright (c) 2007 Baptiste Coudurier 
 *
 * VC-3 encoder funded by the British Broadcasting Corporation

I hear rumors of assorted other pieces receiving formal funding but many details seem to be kept hush.

Jeff’s post has a lot of other interesting money-use cases. However, many of them seem to have the same root requirement: A central FFmpeg organization. We don’t have one. It would be nice to have one, and believe me, I’ve done research on the matter. There are basically 2 options:

  1. Set up our own 501(c)(3) non-profit organization
  2. Join up with another non-profit organization that specializes in being an umbrella organization, thus bestowing individual open source software projects with the benefits of 501(c)(3) status without all the administrative overhead

It was last year about this time that I was very seriously looking into starting a nonprofit organization for the FFmpeg project. First, I purchased Nonprofit Kit For Dummies. This is a very informative and thorough book. At the same time, I attended the Craigslist Foundation Nonprofit Boot Camp. That was pretty useless all around. Both the book and the boot camp made me recognize that I was not up to the task of establishing my own nonprofit organization. Essentially, setting up a 501(c) organization is the single most complex thing you can possibly endeavor to do under the U.S. tax code. This makes sense when you think about it — if you’re going to get away with not paying taxes on income, the government is going to make you toil for the privilege.

So the next option is fiscal sponsorship (the umbrella organization thing). I have tried to engage several such organizations about sponsorship. Short story: I can’t get any of them to even talk to us. I think we might have a reputation. Something to do with our cavalier attitude towards intellectual property matters.

Another problem is that a 501(c)(3) organization needs something called a “board” which is a group of people in charge of the racket. While I might be able to do the legwork for setting up the org (and I am probably the only person who would take the initiative on the matter), I fear that it might be seen as a conflict of interest, due to my day job, for me to be directly involved in its operation. That means I would need to find at least 3 people as crazy as I am who would serve on the board. For bureaucratic purposes, it would probably be easiest if they were American citizens, though that’s not a hard and fast requirement.

I’ve always thought that nonprofit status could help FFmpeg leverage money much more effectively. But for now, that remains an elusive dream.

Posted in Open Source Multimedia | 6 Comments »

JavaFX and On2 TrueMotion

August 16th, 2008 by Multimedia Mike

Have you heard of Sun’s JavaFX? It’s due out later this year and is allegedly positioned to compete in the RIA space. It might be pertinent to mention that I work on a competing technology. Anyway, the reason I bring this up is that I recently learned that On2 is reported to be supplying JavaFX with video codec technology. According to “Sun Adds Comprehensive Video Capabilities to Ubiquitous Java Platform with On2 Technologies,” Sun licensed On2′s “TrueMotion” codec. I’m not entirely sure what codec they’re talking about and I can’t quite find any solid details. On2′s site seems to classify TrueMotion as encompassing both VP6 and VP7. I’m always surprised to hear the name TrueMotion since I thought that went away after the Duck Corporation morphed into On2. But the VP* series seems to be interchangeable with TrueMotion, just for extra confusion.

Who knows? Maybe they actually are using the classic Duck TrueMotion video codec in JavaFX.

Curiously, there is no word on what JavaFX will use for audio. Maybe logarithmic PCM in au/snd files?

Posted in Multimedia PressWatch, On2/Duck | 4 Comments »

200 FATE Tests

August 10th, 2008 by Multimedia Mike

FATE just hit 200 individual test specs. I added a bunch of new H.264 tests that Michael has recently fixed. FFmpeg now perfectly decodes 115/136 of the base H.264 conformance vectors (and a bunch of the extended vectors as well, but I still haven’t gotten to those yet).

Thanks again to Suxen Drol for working on the Electronic Arts family of FMV formats. Thanks to his efforts, FFmpeg can now decode DCT files such as the ones found in Need for Speed II for the PlayStation:


Need 4 Speed -- Jaguar

The file has to come from the Sony PlayStation version since DCT files contain data that is designed to be fed directly into the PS1′s Motion Decoder hardware.

Also, Suxel drol wrote the video decoder for the TGV format, most notably used for FMV in the Origin game Privateer 2: The Darkening:


Privateer 2: The Darkening

Posted in FATE Server, Open Source Multimedia | No Comments »

« Previous Entries