Category Archives: FATE Server

Constantly Compiling Compilers

At the outset of this FATE journey, I pondered how frequently I ought to update gcc from SVN and put new experimental gcc versions into service. At that point, I had some logistical difficulties in upgrading compilers due to my computer setup which made it a serious chore. But it’s much easier to do now– at least for x86_32 and x86_64, I know how to build one compiler that can build both binaries. Further, the C compiler only takes about 45 minutes to compile on my 2.13 GHz Core 2 Duo, leveraging both cores, and barring any other CPU activity.

So, if you know me, you should know that right about now I’m thinking… automated script! Here’s the pitch:

  • Write a new Python script that runs continuously which performs the following steps:
  • ‘svn update’ of my local copy of gcc-svn
  • If the SVN number is higher than the latest build, automatically build a new version and install it in its own directory
  • Wait 24 hours and repeat

This will run on my main FATE machine, so run the script at a low priority so it doesn’t take time away from the more important build/test operations.

So that seems straightforward enough, especially drawing on the experience I have with FATE already (this is basically a simpler version of the main FATE script). For bonus points, the script should also put the freshest compiler into service automatically so I don’t have to be bothered with the tedious, error-prone process. This will be a bit trickier, but it should be possible:

  • Contact the primary FATE MySQL database @ fate.multimedia.cx and update the relevant records in the configuration table.
  • Signal the FATE script to start using the new compiler versions. In the present arrangement, while I have 2 scripts in order to leverage 2 CPU cores, 1 of the scripts is in charge of the 2 gcc-svn configuration for x86_32 and x86_64. The compiler path is specified in a Python configuration file. I may have to modify FATE a little bit so that the script checks some independent file for the compiler location. Then, this new script can update that location.

There is also the matter of storing so many old gcc-svn compiler versions. I’ll likely clean that up manually at various intervals but its not terribly pressing; the main FATE build/test machine inherited a 400 GB hard drive when it went into service (which, you’ll be pleased to hear, is partially used in performing backup duty for the various multimedia.cx domains).

TMV In FFmpeg

I’m pretty sure that Trixter is equally honored and disturbed that Daniel Verkamp has contributed a TMV playback system to FFmpeg.


Neo from "The Matrix", transcoded into TMV format
VLC playing the8trix.tmv transcoded by FFmpeg to PNG-in-AVI

To review, TMV started out as a joke, or perhaps a dare. It was a challenge to try to play 30fps full motion video on an original PC. “Original PC” is defined as 4.77 MHz 8088 CPU, CGA graphics, and original 8-bit Sound Blaster.

For further creepiness, Trixter, here is a new FATE test which will guarantee that FFmpeg reliably decodes TMV files on dozens of different CPU/OS/compiler configurations.

Speaking of FATE, I have neglected to mention that MichaelK has kindly contributed continuous testing resources for NetBSD and OpenSolaris, both on x86_32. Further, Måns continues to work his little ARM-based Beagle Board to the bone and has added more compiler configurations.

Here are some more new FATE tests (thanks to Vitor for adding specific tests to the FATE Test Coverage page):

One more thing– can anyone make FFplay compile under Mac OS X? I thought I succeeded once but now I can’t seem to do it. I would like to take screenshots as seen above using FFplay. But I can’t get the program to work correctly anywhere. Maybe it needs to be formally deprecated… or I need to think of a way to continuously test it via FATE.

Last Performance Smackdown For Awhile

This is getting arduous. I think this will be my last performance smackdown for awhile. First off, I put the latest in the icc 10.1 series — 10.1.022 — into FATE for both x86_32 and x86_64. It seems to work quite well with the 32-bit version having a little trouble with the regression suite; 64-bit version passes all of our tests.

For this test, I decided to use a much shorter video. The file in question has ~10700 frames of MPEG-4 part 2 video at 704×400, along with MP3 audio. The x86_32 performance trend shapes up precisely as we have seen in previous tests, and with a file that takes 1/10 the time to decode. FFmpeg SVN revision is 18737.


32-bit performance comparison, 2009-05-04

As usual, all handcrafted ASM optimizations are disabled. The x86_32 configurations were built with –march/–cpu equal to core2 where available, else pentium4 where available.

Here is the 64-bit chart. It must be noted that FFmpeg compiled with 11.0.083 did not decode the file correctly.


64-bit performance comparison, 2009-05-04

Update: I finally got the dark horse contender — LLVM — to compile at SVN 70961 for x86_64. Out of 2 runs with this same file, it posts a best time of 33.6 seconds.

The differences look severe, but they are actually within a few seconds of each other. And notice that all 64-bit configurations are demonstrably speedier than all 32-bit configurations.

Somehow, it’s only now as I prepare to publish this entry that I realize something amiss– how did my current gcc-svn build manage to build FFmpeg when FATE can’t do the same? It must be the configure options.

See Also:

Latest Compiler News

I’ve been doing compiler stuff tonight:

  • Thanks to Carl Eugen Hoyos as well as my compiler contact inside Intel for advising me on how to procure icc version 11.0.083 (vs. .081 previously) along with an unlimited, non-commercial compiler license. Looks like I won’t have to worry about the 31-day limit now (though there might still be a problem with the Mac OS X version). Further, the 32-bit compiler runs from the 64-bit kernel prompt (I am trying to move away from the 32-bit chroot setup and am meeting with some success). Both 32- and 64-bit versions are now in FATE.
  • After a brief respite following the release of gcc 4.4.0, I have updated the gcc SVN snapshots and reinstated the configurations tracking the FFmpeg build status on experimental gcc 4.5 builds. I’m especially proud, though, that I managed to build one C compiler binary that runs under 64-bit Linux but can build both 32- and 64-bit binaries.
  • A lot of people wonder about this, so I wanted to make it known that I have been briefed on how to use LLVM, the rising star of compiler technology. Thus far, I have not been able to create a build that compiles FFmpeg. I hear that they’re working on it.

New performance smackdowns to come, at least for those that can currently build FFmpeg (the current rev of gcc 4.5-SVN, rev 147090, isn’t doing so well– failing across platforms).