For the most part, I haven’t been able to abide the notion of Twitter, where people “micro-blog” little messages of up to 140 characters each. It just seems like a bunch of “look at me” nonsense with no real purpose. But then, I used to think the same about blogs in general until I found a few interesting blogs. Lately, I have finally found a few interesting people to follow on Twitter.

Then I read mentions of “Twitter clients” and quickly realized that there is an entire software ecosystem around these little messages. Indeed, there is an officially sanctioned HTTP/REST-based API for writing your own client apps.

Naturally, where I’m going with this is: Would it be useful to adapt FATE to post, ahem, “tweets” regarding state transitions (something either broke or got fixed with an individual build)? Would anyone care, i.e., does anyone already actively follow Twitter? I’m getting close to the point where I believe I can implement an email notification system, most likely to a separate mailing list. But this new channel might not be too difficult to implement at the same time. (Actually, I’m still trying to figure out from the documentation whether or not it’s possible to post a new message through the API; I can’t find the right function, or perhaps I just don’t understand all the Twitter-specific jargon yet.)

This is a little more outlandish, but as I was looking at a list of tweets today, I suddenly wondered about the possibility of sending encrypted messages through such a channel. I’m not the only person who was curious. This person beat me to the brainstorm, even going so far as to hack up a proof of concept that encodes a message of arbitrary length into multiple tweets.

Even though I have been studying and working on multimedia technology since 2000 — reverse engineering, documenting, and reimplementing a variety of audio and video codecs — I didn’t actually begin to understand why various algorithms achieved their compression until about 2003. I’m just like that — I study the practice first, and then the underlying theory eventually becomes clear to me (maybe; it has been 9 years and I still couldn’t explain everything about the discrete cosine transform if you asked).

I happened to be looking back over the ZMBV (DOSBox) video codec today. Read the rest of this entry »

I have been studying multimedia technology since 2000. It has been a pretty chaotic technological landscape. People who wanted to publish video on the web wondered what format to use (and occasionally sought my advice). Various fiefdoms arose around Microsoft, Apple, and Real, all hoping to claim the mantle of the standard web video format. Somewhere along the line, Macromedia “accidentally” established a standard web video format via the Flash Player (now Adobe’s).

A few years ago, Adobe (my employer, BTW) upgraded the video support in Flash Player to use the same video format that happened to sit at the top of QuickTime’s codec heap: QT-MP4/H.264/AAC. A few days ago, Microsoft announced the beta of Silverlight 3, which contains support for the same formats. After absorbing that information, it took a few days for the next thought to coalesce in my mind:

We have a standard multimedia format.

All the big players support the same multimedia stack (I think even Real Player supports the same stack). I know that’s dismaying to certain elements of the free software community who insist that Xiph’s multimedia stack is the “standard” (really! there are blessed RFCs to back it up and everything); you may not like it, but that’s the way it is:

• QT-MP4 is the standard container format, not Ogg
• H.264 is the standard video codec, not Theora (or dirac)
• AAC (and also MP3, for historical purposes) is the standard audio codec, not Vorbis

Sure, you may, in principal, have to send a dollar or 2 over to the Patent Illuminati (though highly unlikely). But it’s either that, or, you know, not have a standard video format. (And remember, the HTML5 video tag is not coming to save you.)

At least the free software enthusiast can take comfort in knowing that open source (L/GPL) efforts such as FFmpeg and x264 aim to create the very best tools that anyone can possibly use to create these formats.

Addendum: Now that I think about it, I don’t necessarily know if Silverlight 3 will transport H.264 and AAC inside of a QT-MP4 container or somehow pack it into an ASF file. That would be interesting to find out, though I have read (possibly uninformed) blog chatter excited about being able to stream the same file through Flash and Silverlight.

FFmpeg has been accepted as a mentoring organization in the Google Summer of Code 2009 program. This is the program’s 5th year and FFmpeg’s 4th year in the program. Thanks as always to the big ‘G’.

To refresh, the GSoC pays students $4500 for a summer’s worth of work on qualified open source projects. FFmpeg is one of those projects. FFmpeg team members mentor various multimedia-related projects. Our current project proposals are listed in the wiki. And, as with the previous 2 years, we have some stringent requirements for our students– namely, a student must complete a small FFmpeg task, refining the solution based on critical feedback until it is part of the mainline codebase.

I personally haven’t committed to any mentor possibilities. I wonder if I should propose the Theora encoder again, which didn’t quite pan out last year? The thing is practically written already in my mind.

Someone recently asked on the ffmpeg-devel mailing list for nightly binary builds of FFmpeg and its associated libraries to be made available for download. The idea seemed to gain traction on the list so I began thinking of how we can make this a reality. Since FATE clients are already building FFmpeg binaries day and night, it’s not a huge leap to consider extending the infrastructure slightly to include packaging the binaries and sending them to a central server. So let’s talk process:

• After building binaries as part of FATE, make a compressed bundle (.tar.bz2 for Unix, .zip for Windows) and ship the package right here to multimedia.cx. Hey, I have the space to spare (an alleged 500 GB with the current hosting plan, of which I am currently barely using 6 GB, and nearly half of that is FATE data). I also have FTP account options which should facilitate transfer.
• The filename would follow the naming convention of “ffmpeg-bin-svn-OS-arch-compiler.ext”. So, for example, the build of SVN 18005 from Linux / x86_32 / gcc 4.2.4 would be: ffmpeg-bin-svn18005-linux-x86_32-gcc-4.2.4.tar.bz2.
• What should be in the package? The FFmpeg binary programs and the associated libraries, of course. But in what hierarchy? A flat hierarchy doesn’t strike me as a good idea (unzip and potentially trash existing files in your current directory). Instead, package the files in, e.g., ffmpeg-svn18005-linux-x86_32-gcc-4.2.4/.
• There should be a standard, auto-generated README.txt file in the same directory. Actually, perhaps it should be a simple HTML file since I envision it should contain — in addition to basic information such as its revision and the time it was built — a report of which tests passed and which failed, along with links to each report in the FATE database.
• Create a web interface that allows users to browse among the latest FFmpeg binary builds. Out of everything presented in this brainstorm, this is the step that actually gives me some pause since I don’t exactly know how I would implement it off the top of my head.

Feedback welcome. Otherwise, these are likely to be the decisions I run with for the first iteration of this plan, when I get around to it (think of this as the public comment period).

Okay, so you may have noticed that FATE has 3 more machines/platforms from which it is actively accumulating build/test results: FreeBSD/x86_32, DragonFly BSD/x86_32, and Linux/AVR32 (thanks to Michael K. and Måns for running the FATE script on those platforms). I was wondering when a volunteer would step forth to continuously run FATE on some BSD platform.

But the FATE main page is now unbearably unwieldy. When I first put FATE into service, the entire front page was dynamically generated using some horrifically unoptimized queries. A little over a year ago, I rushed a highly naive caching mechanism into production to address this problem. The way it operated was as follows (you might want to be sitting down for this): From a cron job run every 15 minutes, run a Python script that connects to the database server, performs all the queries for fetching data for the main page, and builds a file called main-page-cache.php which is then transferred to the server where it overwrites the old copy. The main index.php script driving the site simply includes the file. Okay, don’t all go contributing this to The Daily WTF at once. Hey, it has performed rather quickly for the last year or so. But it also explains why I couldn’t modify it very easily to allow arbitrary sorting.

Fun fact: At this point, index.php did not strictly need to connect to the database in order to simply render the main page. However, I quickly learned to do a no-op database connection anyway; without it, the end user’s browser would cache the output of the page. I’m sure there must be better browser cache control mechanisms available.

Until I switched web hosts recently, I had to run the Python script on a local machine and scp the results to the server. This explains why, if there was a power failure at home that outlasted the UPS, the main FATE page wouldn’t get updated until I returned home. That problem is solved now by the fact that the cron job and script run on the same server as FATE. At least, the problem was solved until I had to switch back to my prior web host.

After I deployed the caching system, I wondered about ways I could possibly trigger cache page rebuilds automatically after entering new results. Last autumn, I revised the FATE architecture so that results are received through a PHP script. I started to think that, when a configuration sends results through this script, it could update a new PHP cache data file specific to that configuration. The main script could dynamically include a series of these PHP cache files and sort out the data inside.

Then it dawned on me to store the data in an SQLite database with a single, highly non-normalized table. And just when I was about to go to code on that idea, it occurred to me that, as much as I love SQLite, there’s really no reason I can’t put the cache table straight into the main MySQL FATE database. The big advantages will be: No more up-to-15-minute delay before new results show up on main page; and more flexible sorting of the results on the main page.

So the plan goes like this:

1. Create a non-normalized cache table in the main FATE database that includes config ID, machine ID. OS, architecture, compiler string, latest build ID, timestamp build ID was logged, SVN revision it corresponds to, status of the build, number of tests total, number of tests passed.
2. Create a script that performs an initial population of this table based on the configurations marked active and their latest build records in the database. It will be necessary to re-run this script where there are new configurations added to the database, or when configuration data is updated (most often happens when I compile a new gcc from SVN).
3. Modify the data receiver PHP script so that it properly updates the correct row in the table.
4. Go crazy with the main FATE page. Sorting the data by different fields is as straightforward as an ORDER BY clause in the SELECT statement.

The main page should also send a cookie so that the page “remembers” the user’s last (and presumably preferred) sorting order. That assumes that any of FATE’s users actually browse the web with cookies enabled (doesn’t strike me as likely).

Writing out these ideas is useful for motivating further brainstorms. I just realized that I may as well create a simple method for accessing the latest FATE data via HTTP, perhaps output in a CSV format (no XML, thanks). Perhaps others can think of creative ways to interpret the data and act on it. E.g., maybe someone else can figure out creative ways of sending email and IRC notifications before I can get time to solve those problems.

The last few weeks have taught me that all web hosts suck. Ordinarily, I don’t feel it necessary to write a blog post railing and ranting against a company that annoys me; I do my best to let these things go quickly. However, according to the “sucky research method”, WebFaction has thus far managed to escape any negative criticisms. This wouldn’t be such a problem except that they’re a tad smug about it. Further, I think I am having trouble achieving emotional closure and catharsis on this matter since I find no similarly suffering souls out there with whom to commiserate. So it is with a heavy heart that I feel compelled to type out a petty, petulant “WebFaction sucks” post.

Who knows? Maybe I’m just the unluckiest customer a web host has ever had. Read the rest of this entry »

If you’re reading this, the multimedia.cx DNS change has propagated, yet again, for another hosting service. But that’s a venomous tale for another blog post. FFmpeg v0.5 hit the web yesterday, right about the time that my new hosting service started having serious problems. Thus, I was unable to observe the occasion at the time.

But now I’m able to let it sink in a bit more… wow. I can’t even remember the last time we had a release and to be honest, I don’t really want to look it up because it’s a little embarrassing to think about. Thanks to all my co-devs who made the release what it is. But thanks especially to Diego Biurrun who stuck his neck out and pushed hard for this release effort starting a little over a month ago. Somehow, we got the release out.

Rest assured, we are taking notes this time around in order to make the process easier next time around. This is the dawn of a new era of open source multimedia (here’s hoping, anyway).

How did I become the benchmark peon? Oh right, I actually dared to put forth some solid benchmarks and called for suggestions for possible improvements to the benchmark methodology. This is what I get.

Doing these benchmarks per all the suggestions I have received is time-consuming and error-prone. But if you know anything about me by now, you should know that I like automating time-consuming and error-prone tasks. This problem is looking more and more like a nail, so allow me to apply my new favorite hammer: Python!

Here’s the pitch: Write a Python script that iterates through a sequence of compiler configurations, each with its own path and unique cflags, and compiles FFmpeg. For each resulting build, decode a long movie twice, tracking the execution time in milliseconds. Also, for good measure, follow Reimar’s advice and validate that the builds are doing the right thing. To this end, transcode the first 10 seconds of the movie to a separate, unique file for later inspection. After each iteration, write the results to a CSV file for graphing.

And here’s the graph:

Look at that! gcc 4.3.2 still isn’t a contender but gcc 4.4-svn is putting up a fight.

Here are the precise details of this run:

• Movie file is the same as before: 104-minute AVI; ISO MPEG-4 part 2 video (a.k.a. DivX/XviD) at 512×224, 24 fps; 32 kbps, 48 kHz MP3
• This experiment includes gcc 4.4.0-svn, revision 143046, built on 2009-01-03 (I’m a bit behind)
• All validations passed
• Machine is a Core 2 Duo, 2.13 GHz
• All 8 configurations are compiled with –disable-amd3dnow –disable-amd3dnowext –disable-mmx –disable-mmx2 –disable-sse –disable-ssse3 –disable-yasm
• icc configuration compiled with –cpu=core2 –parallel
• gcc 4.3.2 and 4.4.0-svn configurations compiled with -march=core2 -mtune=core2
• all other gcc versions compiled with no special options

See Also:

• Next smackdown in the series
• Previous smackdown in the series

What’s in store for round 4? It sure would be nice to get icc 11.0 series working on my machine for once to see if it can do any better. And since I have the benchmark framework, it would be nice to stuff LLVM in there to see how it stacks up. I would also like to see how the various builds perform when decoding H.264/AAC. The problem with that is the tremendous memory leak that slows execution to a crawl during a lengthy transcode. Of course I would be willing to entertain any suggestions you have for compiler options in the next round.

Better yet, perhaps you would like to try out the framework yourself. As is my custom, I like to publish my ad-hoc Python scripts here on my blog or else I might never be able to find them again.

Read the rest of this entry »

I heard an old, familiar song on the radio this morning. But something about it was off, and I knew what. I found myself yelling at the radio, “Use a higher bitrate!” For you see, the chorus of the song exhibited something that sounded like the notorious “underwater” artifact in MP3 when encoding with too low a bitrate.

I remember first hearing perhaps 10 years ago that radio stations were starting to move all of their music to MP3 (prior to that, I remember hearing that some would have a stack of about 10 CD players with music queued up; who really knows? And I’m sure varying radio stations use different equipment and setups). I just assumed that a radio station would use the highest bitrate possible. Perhaps this particular encoding was a leftover from when the radio station first moved to MP3 (the song itself was from 1995), when they assigned an intern to use some shareware encoder that was only capable of 96 kbps MP3.

I know I can’t be the only multimedia geek who gets frustrated at seeing sub-optimality deployed in the world at large. I remember staying at a hotel during Christmas of 2000 (the same year I was just starting to study multimedia) where the in-hotel movie preview system through the TV displayed horrible blocking artifacts. At the time, I only vaguely understood what could have been going on.