Category Archives: Open Source Multimedia

News regarding open source multimedia projects.

AAC Decoder Is In!

It certainly has been a long journey for native Advanced Audio Coding (AAC) in FFmpeg. It started with a Google Summer of Code project back in FFmpeg’s inaugural FFmpeg SoC season (2006). It went unfinished. Since then, many people have endeavored to fix it up to the point where it can be included into the mainline. But it was Robert Swain who persevered toward the end goal. And now look:

$ ffmpeg -formats
[...]
Codecs:
 D V    4xm             4X Movie
 D V D  8bps            QuickTime 8BPS video
 D A    8svx_exp        8SVX exponential
 D A    8svx_fib        8SVX fibonacci
 D A    aac             Advanced Audio Coding
[...]

Robert profiled the new AAC decoder to be significantly faster than the libfaad, the prevailing AAC decoding solution in the open source community. Further optimization work is ongoing, as is support for more advanced coding modes. Currently, the decoder only deals with low complexity (AAC-LC), the most common variant you are likely to encounter.

And of course, thanks also to Robert for creating more FATE work for me. I can’t avoid the problem of testing perceptual audio decoders for much longer.

Baldur’s Task

I have a lot of experience consuming multimedia on Linux using open source software. But I have little experience generating or transcoding multimedia within the same parameters. So, much like I experienced with the MOOBEX exercise, I decided to move out of my comfort zone and work on transcoding for once.

Fortunately, I have a concrete task– a colleague who is interested in game preservation wanted to know about transcoding the Interplay MVE files from games such as Baldur’s Gate into a format a little more modern. So let’s see what we can do about using FFmpeg to transcode to MP4/H.264/AAC.


Baldur's Gate logo

I suspect the steps will look like this:

  • Download and install libfaac, as Kostya’s AAC encoder is not in the mainline tree yet
  • Download and install a recent snapshot of x264
  • Update to the latest SVN snapshot of FFmpeg and compile it with libfaac and x264 support
  • Transcode away, preferably with some manner of batch transcode process

Let’s see if it’s that easy.

This is the first time I have ever worked with x264. I decided to install from source. It has trouble finding and using the yasm assembler installed on the system. No matter, I’ll be patient for the test encodings. I finally got everything compiled together and figured out what options FFmpeg wanted (‘-acodec libfaac -vcodec libx264’) and I let it rip…

…and the whole thing blows up. Segfault down in the x264 library.

Between this and the MOOBEX exercise and the library problems on my little work project, I can’t help but take a dim view of multimedia support on Linux this week. Which is especially depressing since I sort of consider multimedia on Linux to be my beat.

Dog food sucks. But on the plus side, Robert finished committing AAC-LC decoding support to FFmpeg.

Introducing MOOBEX

I was sitting at The Linux Foundation’s Collaboration Summit last April in Austin, Texas, USA pondering the state of multimedia usability on Linux. I realized that I live a somewhat sheltered computing existence vis-à-vis consuming multimedia because I happen to know a little bit more about it than most users. Then I got to wondering what the typical uninitiated user experiences when trying to play multimedia in Linux these days.

That’s the moment when I conceived MOOBEX: Multimedia Out-Of-Box EXperience. (Initially, I called it MOOB. But I wisely googled to see if that already had some meaning attached to it. I’m glad I did, and I’ll leave it to the reader to learn about the colloquial connotations of that word if they feel so inclined.)

For MOOBEX, I raided my vast personal multimedia archive and assembled a small suite of what I consider to be representative multimedia types. Then I developed a strict process for testing the multimedia capabilities of various stock Linux distributions from the perspective of a new Linux user. I tested the latest versions of 5 Linux distributions for the first pass at this. These include (x86_32 in all cases):

Fairness (and general curiosity) dictates that I give equal treatment to non-Linux operating systems. Thus, this initial pass also covers:

Executive Summary: Mandriva is absolutely remarkable with its out-of-the-box multimedia capability. Ubuntu is almost as good– while it isn’t too capable out of the box, it does a fine job of holding the user’s hand while installing required multimedia software on demand. The custom Xandros install on the Eee PC is also very good, albeit with a few holes. OpenSUSE isn’t too capable. Fedora Core’s multimedia support is deplorable and stultifyingly patronizing to boot (but, to be fair, somewhat improved from 2 years ago).

Apple’s and Microsoft’s operating systems perform as expected, able to play their respective company’s technologies perfectly out of the box but not being able to deviate from that, save for Adobe Flash.

I will have more analyses and opinions about my findings soon. In the meantime, the raw data is available in the MultimediaWiki. Also, be advised: No one can play an AIFF file anymore (except Apple). Maybe I’m the only person who finds this surprising; AIFF files weren’t that uncommon, were they?

Just as I was finishing this first round of reports, I realized that I named one sample girls-just-wanna-cook.rm. Wow, that could get me in trouble. I took the FUN_RM_32.rm sample from the samples archive — a 15-second sample of Cyndi Lauper’s timeless “Girls Just Wanna Have Fun” and renamed it to denote that it uses the RealAudio Cooker ‘COOK’ codec. Oops. Well, I’m not changing all those reports now.

The Intersection of Code And Money

Some colleagues have advised me to lend my commentary regarding Jeff Atwood’s much-discussed piece entitled “Is Money Useless to Open Source Projects?” (or, as the Linux Hater put it, “You can’t even pay them to not suck“). I think I was supposed to argue the position that FFmpeg provides an efficient mechanism to convert raw currency into high quality multimedia source code. But I’m not sure I make that case or refute Jeff’s piece; in fact, I largely agree with his post. However, I can corroborate that FFmpeg does sometimes benefit from raw currency.

Let’s talk about the most visible example of the cash->code thesis as it relates to FFmpeg: Google’s Summer of Code sponsorship of FFmpeg. In 3 years, FFmpeg has earned 22 combined project slots; each slot represents a transfer for US$5000 from Google -> people involved in the project. Wow, this is the first time I have realized that Google has, to date, committed over 100 G’s to our silly little multimedia project. What has it gotten for that investment? Well, it got the foundation that made YouTube possible. What has FFmpeg been able to do with all that green? Out of 13 projects in 2006 and 2007, only 4 have currently reached acceptable status and are part of the mainline code tree.

Fortunately, this year, we have imposed much more stringent standards for the projects; students must complete their features to a level that they can be incorporated into the main tree by the end of the project. This week marks the end of GSoC 2008, so we will see how that turns out. If you have been following the FFmpeg mailing lists, you no doubt noticed a flurry of activity as students race to get all parts of their various projects approved for inclusion into the mainline tree.

Apart from GSoC, there are tales of different parts of FFmpeg development being actively funded by various organizations. The file dnxhdenc.c carries the following header:

 * VC3/DNxHD encoder
 * Copyright (c) 2007 Baptiste Coudurier 
 *
 * VC-3 encoder funded by the British Broadcasting Corporation

I hear rumors of assorted other pieces receiving formal funding but many details seem to be kept hush.

Jeff’s post has a lot of other interesting money-use cases. However, many of them seem to have the same root requirement: A central FFmpeg organization. We don’t have one. It would be nice to have one, and believe me, I’ve done research on the matter. There are basically 2 options:

  1. Set up our own 501(c)(3) non-profit organization
  2. Join up with another non-profit organization that specializes in being an umbrella organization, thus bestowing individual open source software projects with the benefits of 501(c)(3) status without all the administrative overhead

It was last year about this time that I was very seriously looking into starting a nonprofit organization for the FFmpeg project. First, I purchased Nonprofit Kit For Dummies. This is a very informative and thorough book. At the same time, I attended the Craigslist Foundation Nonprofit Boot Camp. That was pretty useless all around. Both the book and the boot camp made me recognize that I was not up to the task of establishing my own nonprofit organization. Essentially, setting up a 501(c) organization is the single most complex thing you can possibly endeavor to do under the U.S. tax code. This makes sense when you think about it — if you’re going to get away with not paying taxes on income, the government is going to make you toil for the privilege.

So the next option is fiscal sponsorship (the umbrella organization thing). I have tried to engage several such organizations about sponsorship. Short story: I can’t get any of them to even talk to us. I think we might have a reputation. Something to do with our cavalier attitude towards intellectual property matters.

Another problem is that a 501(c)(3) organization needs something called a “board” which is a group of people in charge of the racket. While I might be able to do the legwork for setting up the org (and I am probably the only person who would take the initiative on the matter), I fear that it might be seen as a conflict of interest, due to my day job, for me to be directly involved in its operation. That means I would need to find at least 3 people as crazy as I am who would serve on the board. For bureaucratic purposes, it would probably be easiest if they were American citizens, though that’s not a hard and fast requirement.

I’ve always thought that nonprofit status could help FFmpeg leverage money much more effectively. But for now, that remains an elusive dream.