The IETF has recently put out a request for an audio codec. This may strike some of you as remarkable that anyone would need another audio codec since, at the time of this writing, we have cataloged 137 audio codecs via the MultimediaWiki. You have to give the request some attention, though– it acknowledges that there are already lots and lots of audio codecs in existence and explains why each category is unsuitable to the goals of the request. I’m not going to be the one to audit every one of those 137 codecs and identify why each is unsuitable for the outlined goals.

I am a bit concerned about some of their stated goals, such as the very first one: “Designing for use in interactive applications (examples include, but are not limited to, point-to-point voice calls, multi-party voice conferencing, telepresence, teleoperation, in-game voice chat, and live music performance).” Generally, one of those examples is not like the others (unless, perhaps, “live music performance” refers to a cappella singing. Then again, the request later states that optimizing for very low bitrates (2.4 kbps and lower) is out of scope.

I accidentally used the main ‘ffmpeg’ binary as an input to itself. Its best guess is that it’s an MP3 container with MPEG-1, layer 1 audio data:

[NULL @ 0x1002600]Format detected only with low score of 25, misdetection possible!
[mp1 @ 0x1003600]Header missing
    Last message repeated 35 times
[mp3 @ 0x1002600]max_analyze_duration reached
[mp3 @ 0x1002600]Estimating duration from bitrate, this may be inaccurate
Input #0, mp3, from 'ffmpeg_g':
  Duration: 00:03:20.29, start: 0.000000, bitrate: 256 kb/s
    Stream #0.0: Audio: mp1, 32000 Hz, 2 channels, s16, 256 kb/s
At least one output file must be specified

What an Easter egg it would be if the compiled binary could actually decode to something — anything — valid.

Last week brought us a lot of news in the web browser space: Mozilla released Firefox 3.6 (nice fullscreen video, BTW, especially on Linux); YouTube and Vimeo grabbed headlines by announcing HTML5 video support for their video sites.

I resolved a few months ago to not bother reading so many tech news sites since they consist of 99% misinformed drivel, and I’m a happier person for that decision. But when there’s big news that can be seen as tangentially related to what I do at my day job, it gets hard to resist.

From everything I read, there was surprisingly little Flash hatred in the wake of these announcements. Really, the situation just erupted into an all-out war between the devotees of Firefox (and to a lesser extent, Opera) and supporters of Google (and to a lesser extent Apple and their Safari browser). It gets boring and repetitive in a hurry when you start reading these discussions since they all go something like this:

As you can see from the infographic, at least both sides can agree on something. I would also like to state my emphatic support for Mozilla’s principled, hardline stance against the MPEG stack for HTML5 video. Please don’t budge on your position. Stand firm on the moral high ground.

That graphic is just the beginning; there are so many problems with HTML5 video that it’s hard to know where to even begin. That’s why I need to remember to just laugh gently at its mention and move along. I only get a headache trying to understand how HTML5 video could ever have the slightest chance of mattering in the grand scheme of things.

However, a pleasant side effect of this attention is that more and more people are actually being exposed to the video tag. One nagging detail people invariably notice is that the video tag performs exceptionally poorly, likely because browsers have to deal with the exact same limitations that the Flash Player does, namely, converting decoded YUV data to RGB so that it can be plopped on a browser page. And if you try to claim that you can just download the media and use a standalone player, you continue to miss the entire point of web video.

Another aspect I have to appreciate about the debate surrounding HTML5 video is the way that it brings out the positive spirit in people. Online discussions are normally overwhelmingly negative. But advocates of the HTML5/Xiph approach truly believe this could all work out: If Apple decides to adopt the Xiph stack, and if some benevolent hardware company would churn out custom ASICs for decoding Xiph codecs, and if those ASICs were adopted in next quarter’s array of mobile computing devices and netbooks, and if Google transcodes their zillobytes of YouTube videos to the Xiph stack, and if Google throws the switch and forces the 60% of IE-using stragglers to either change browsers or go without YouTube, and if Google thereby forgoes many opportunities to monetize their videos, then absolutely! HTML5 video could totally unseat Flash video.

Okay, that’s it for me. I’m going to go back to ignoring the insular, elitist tech world at large except for the few domains in which I have some influence.

See Also:

• The last time I put any effort into evaluating the HTML5 video tag

Pursuant to my rant on the futility of comparing, performance-wise, the output of various compilers, I wholly acknowledge the utility of systematically benchmarking FFmpeg. FATE is not an appropriate mechanism for doing so, at least not in its normal mode of operation. The “normal mode” would have each of every configuration (60 or so) running certain extended test specs during every cycle. Quite a waste.

Hypothesis: By tracking the performance of a single x86_64 configuration, we should be able to catch performance regressions in FFmpeg.

Proposed methodology: Create a new script that watches for SVN commits. For each and every commit (no skipping), check out the code, build it, and run a series of longer tests. Log the results and move on to the next revision.

What compiler to use? I’m thinking about using gcc 4.2.4 for this. In my (now abandoned) controlled benchmarks, it was the worst performer by a notable margin. I’m thinking that the low performance might help to accentuate performance regressions. Is this a plausible theory? 2 years of testing via FATE haven’t revealed any other major problems with this version.

What kind of samples to test? Thankfully, Big Buck Bunny is available in 4 common formats:

• MP4/MPEG-4 part 2 video/AC3 audio
• MP4/H.264 video/AAC audio
• Ogg/Theora video/Vorbis audio
• AVI/MS MPEG-4 video/MP3 audio

I have the 1080p versions of all those files, though I’m not sure if it’s necessary to decode all 10 minutes of each. It depends on what kind of hardware I select to run this on.

Further, I may wish to rip an entire audio CD as a single track, encode it with MP3, Vorbis, AAC, WMA, FLAC, and ALAC, and decode each of those.

What other common formats would be useful to track? Note that I only wish to benchmark decoding. My reasoning for this is that decoding should, on the whole, only ever get faster, never slower. Encoding might justifiably get slower as algorithmic trade-offs are made.

I’m torn on the matter of whether to validate the decoding output during the benchmarking test. The case against validation says that computing framecrc’s is going to impact the overall benchmarking process; further, validation is redundant since that’s FATE’s main job. The case for validation says that since this will always be run on the same configuration, there is no need to worry about off-by-1 rounding issues; further, if a validation fails, that data point can be scrapped (which will also happen if a build fails) and will not count towards the overall trend. An errant build could throw off the performance data. Back on the ‘against’ side, that’s exactly what statistical methods like weighted moving averages are supposed to help smooth out.

I’m hoping that graphing this idea for all to see will be made trivial thanks do Google’s Visualization API.

The script would run continuously, waiting for new SVN commits. When it’s not busy with new code, it would work backwards through FFmpeg’s history to backfill performance data.

So, does this whole idea hold water?

If I really want to run this on every single commit, I’m going to have to do a little analysis to determine a reasonable average number of FFmpeg SVN commits per day over the past year and perhaps what the rate of change is (I’m almost certain the rate of commits has been increasing). If anyone would like to take on that task, that would be a useful exercise (‘svn log’, some text manipulation tools, and a spreadsheet should do the trick; you could even put it in a Google Spreadsheet and post a comment with a link to the published document).

You know what? I’m tired of being the benchmark bitch.

I have come to realize that benchmarks are exclusively the domain of people who have a vested interest in the results. I’m not financially or emotionally invested in the results of these compiler output performance comparisons; it was purely a matter of academic curiosity. As such, it’s hard to keep the proper motivation. No matter how carefully I set up a test and regardless of how many variables I control, the feedback is always the same: “Try tweaking this parameter! I’m sure that will make my favorite compiler clean up over all the other options!” “Your statistical methods are all wrong!” “Your graphs are fraudulently misleading.” Okay, I wholeheartedly agree with that last one, but blame OpenOffice for creating completely inept graphs by default.

Look, people, compilers can’t work magic. Deal with it. “–tweak-optim-parm=62″ isn’t going to net the 10% performance increase needed to make the ethically pure, free software compiler leapfrop over the evil, proprietary compiler (and don’t talk to me about profile-guided optimization; I’m pretty sure that’s an elaborate hoax and even if it’s not, the proprietary compilers also advertise the same feature). Don’t put your faith in the compiler to make your code suck less (don’t I know). Investigate some actual computer science instead. It’s especially foolhardy to count on compiler optimizations in open source software. Not necessarily because of gcc’s quality (as you know, I think gcc does remarkably well considering its charter), but because there are so many versions of compilers that are expected to compile Linux and open source software in general. The more pressing problem (addressed by FATE) is making sure that a key piece of free software continues to compile and function correctly on a spectacular diversity of build and computing environments.

If anyone else has a particular interest in controlled FFmpeg benchmarks, you may wish to start with my automated Python script in this blog post. It’s the only thing that kept me sane when running these benchmarks up to this point.

I should clarify that I am still interested in reorganizing FATE so that it will help us to systematically identify performance regressions in critical parts of the code. The performance comparison I care most about is whether today’s FFmpeg SVN copy is slower than yesterday’s SVN copy.

See Also:

• The final entry in the compiler smackdown series

Update: I have to state that I’m not especially hurt by any criticism of my methods (though the post may have made it seem that way). Mostly, I wanted to force myself to quit wasting time on these progressively more elaborate and time-consuming benchmarks when they’re really not terribly useful in the grand scheme of things. I found myself brainstorming some rather involved profiling projects and I had to smack myself. I have far more practical things I really should be using my free time for.

It’s time to do a new compiler smackdown for a few reasons:

1. It has been quite awhile since the last one.
2. I received a request to know how icc 11.1 measured up.
3. I wanted an excuse to post a picture of the GCC cheerleaders.

For this round, I tested x86_64 on my Core 2 Duo 2.0 GHz. I compiled FFmpeg with 6 versions of gcc (including gcc 4.5, svn 156187), 3 versions of icc, and the latest (svn 94292) of LLVM. Then I used the resulting FFmpeg binaries to decode both a Theora/Vorbis video and an H.264/AAC video.

Ogg/Theora/Vorbis, 1920×1080 video, 48000 Hz stereo audio, nearly 10 minutes:

MP4/H.264/AAC: 1280×720 video, 48000 Hz stereo audio, 4.5 minutes:

Wow! Look at LLVM go. I take back all, or at least some, of the smack I’ve typed about it in previous posts. Out of the free compiler solutions, LLVM makes my Theora code suck the least.

Other relevant data about this round:

• FFmpeg SVN 21390 used for this test
• Flags: ‘–disable-debug –disable-amd3dnow –disable-amd3dnowext –disable-mmx –disable-mmx2 –disable-sse –disable-ssse3 –disable-yasm’ used for all configurations; also used ‘–disable-asm’ which might make a lot of those obsolete now.
• gcc 4.3-4.5 used “-march=core2 -mtune=core2″; icc versions used “–cpu=core2 –parallel”

See Also:

• Final wrap-up for this series
• Previous smackdown in the series

It happens on occasion that everyone gushes about a technology that leaves me utterly bewildered– not because I don’t understand what the tech does but because I can’t get it to work at all. It happened the first time I tried out xine and the first time I tried to use x264 for anything serious. But I eventually solved those situations.

Here’s a new problem that has been bugging me: Google Docs, specifically their spreadsheets. They’re so bold as to launch a daily-updated ad campaign of billboards declaring that it’s perfectly plausible to switch an entire office from Microsoft Office software to Google software. For the last few months, I have been using Google Spreadsheets to track a fairly meager amount of data. It hasn’t been going that swell. So whenever I see a fawning article about how Google is totally going to dominate Microsoft using these online apps, I’m left scratching my head and wondering if I’m missing something.

This is my tracking spreadsheet for all my video games. You’re likely to notice that it’s quite taxing on your browser just to load the DOS/Windows sheet which only consists of about 600 rows. This data started life on an OpenOffice Calc spreadsheet before eventually being imported to Google Docs. I should state that I have never been very proficient with spreadsheets. Obviously, all I’m doing here is organizing tables of information with a little coloring. So, no complicated (or even simple) formulas. Maybe it’s all the coloring that throws the system for a loop. Or something about its OOo origins.

For a current test, I downloaded the very latest versions of Firefox (3.6, as of today), Safari, and Chrome on my Atom-based nettop running Windows XP. This is a basic visualization describing how each handles opening my games spreadsheet in Google Docs:

Firefox takes awhile to load the spreadsheet (up to 30s) and pegs one CPU the entire time. But once the spreadsheet loads, the browser chills until there is some more interaction. Safari seems to load a bit quicker but never takes the load off that 1 CPU. The spreadsheet is still a little usable.

Most surprising, however, is that Google’s Chrome, for all intents and purposes, completely falls over on the games spreadsheet. This is using the very latest version on Windows, which I assume is the version that receives the lion’s share of Chrome’s dev resources. Chrome eventually loads the spreadsheet (older versions had trouble even doing that), but can’t seem to scroll through it.

Given all the hype — both within and outside of Google — that I see surrounding Google Docs, I can’t help but wonder if I’m doing something wrong. Am I throwing too much data its way? I’m anything but a typical spreadsheet user so it could very well be that most spreadsheets contain well fewer than 600 lines of data. Somehow, I think it has something to do with having imported the data for an OpenOffice Calc spreadsheet. My data point for this is that another Google spreadsheet that I have maintained from scratch but has grown to a similar size has much less trouble. Further, I generated a 13,000-line CSV file, imported that, and see little difficulty (relative to the games spreadsheet) navigating around. Pro tip: Don’t try to sort 13,000 rows of data in a Google spreadsheet:

Perhaps it’s an unreasonable request. I do know that OpenOffice is able to process the same request in about 2 seconds.

But I digress. I was wondering how Google could possibly claim this is ready for prime-time. Then I realized that Excel spreadsheets are more likely to be thrown at the system. I decided to try exporting the spreadsheet as an Excel spreadsheet (loading the Google spreadsheet in Firefox since that’s the most responsive), then uploading the new Excel spreadsheet.

Success! Firefox browses the spreadsheet much faster and Google Chrome is able to navigate it at all. It’s still a bit sluggish in Chrome but it’s at least a little usable. You know, pursuant to today’s Firefox 3.6 release, I have been reading comments that Chrome still beats Firefox in some artificial JavaScript benchmarks. After this episode, I think this would make a much more useful, real-world JS benchmark.

Through it all, I really want to be able to make use of Google Docs. So far, it has proven very useful as a means of coordination between myself and a bunch of other video game historians. But I was befuddled to no end that I couldn’t get my favorite spreadsheet to work in Google’s own browser.

For reference, here are the old and new spreadsheets for comparison in your browser:

• Old spreadsheet, imported from OpenOffice Calc spreadsheet
• New spreadsheet, exported to Microsoft Excel and re-imported

I bought this VideoNow disc a long time ago. It was during the month of December, but the year is a bit fuzzy. It was probably 2004. I wrote up a brief MultimediaWiki page somewhere along the line and even found a set of free tools for processing the data on these discs. But there was just one minor issue keeping me from investigating this disc thoroughly: How do I read it?

Read the rest of this entry »

Pursuant to some archiving projects I want to conduct, I wanted to evaluate Bzip2 vs. LZMA for compression. I know that the latter is more efficient, size-wise, than the former while generally requiring more time on the compression side. But I wanted to know if the encoding time difference was very severe vs. the space saved. I also wanted to know how the relative decode speed compares.

Methodology: For a number of large files that are each around 1.35 GB, measure the compression speed and ratio and then measure…

You know what? This is the most basic type of profiling experiment to set up and I really don’t feel like describing the process, the hardware used, the variables carefully controlled, or graphing the data. Here’s what I came up with in my tests:

• Bzip2 is 2-2.3x faster to compress than LZMA.
• The Bzip2 files were 15-20% larger than the LZMA files.
• The LZMA files decompressed in nearly half the time of the Bzip2 files.

Conclusion: I’ll be going with LZMA for my long-term archival projects.

Via the MultimediaWiki, Suxen Drol made me aware of recent video codec identified only as SIF1. It seems the codec has been on the radar for a few months now. The decoder source code (Windows) is available for download, as are a bunch of samples. Is anyone interested in writing a proper description for this codec based on the source code? If so, I hope you can understand whatever (human) language the author wrote it in. Here are the core filenames:

$ wc *.cpp
    1185    2217   37009 AdaptFiltrDequant.cpp
    4375   13045  119493 BikubDetcimation.cpp
   14075   42529  533493 DeblokFiltrCompDvij.cpp
    1566    3110   57229 MotionArifmDecomp.cpp
     820    2429   23311 Rgb_viu_kon.cpp
   21588   70557  889709 SifDecompressor.cpp
     118     408    3577 SifEkspotrFunk.cpp
     394    1063   12274 SifFilter.cpp
      96     323    2764 SifFiltrUprDialog.cpp
   44217  135681 1678859 total

Okay, so maybe not all of those filenames are so bad, but I challenge you to decipher many of the identifiers in the source. Also, some of those files are a tad bulky. Large swaths of code are written only in inline MMX. I haven’t seen this since Lagarith. I have this weird feeling that this codec is going to exist in its own little universe for a long time to come even though the author seems to have invested a lot of research into making it competitive with H.264.