Tag Archives: gcc

LLVM Recognition

I did a bunch with compilers in FATE tonight:

  • I formally inducted x86_32 and x86_64 configurations of LLVM for Linux into the FATE farm. I’ve heard that compiler is going to rescue us all someday. For the time being, it is exhibiting some problems with the FFmpeg source code, particularly on 32-bit.
  • I upgraded the gcc used for compiling the Mac OS X versions to gcc 4.0.1 build 5493, packaged with Xcode 3.1.3 which is the latest version per my understanding. I think this is the first time I have bothered to upgrade Xcode since the first time I installed it. Everything still runs smoothly there, both on 32- and 64-bit.
  • I updated the gcc-svn snapshots across x86_32, x86_64, and PowerPC, all on Linux. All 3 configurations are compiling now. That’s the good news. Regrettably, the PowerPC build is doing even worse now (i.e., failing more tests) than the recently released gcc 4.4.0.

Constantly Compiling Compilers

At the outset of this FATE journey, I pondered how frequently I ought to update gcc from SVN and put new experimental gcc versions into service. At that point, I had some logistical difficulties in upgrading compilers due to my computer setup which made it a serious chore. But it’s much easier to do now– at least for x86_32 and x86_64, I know how to build one compiler that can build both binaries. Further, the C compiler only takes about 45 minutes to compile on my 2.13 GHz Core 2 Duo, leveraging both cores, and barring any other CPU activity.

So, if you know me, you should know that right about now I’m thinking… automated script! Here’s the pitch:

  • Write a new Python script that runs continuously which performs the following steps:
  • ‘svn update’ of my local copy of gcc-svn
  • If the SVN number is higher than the latest build, automatically build a new version and install it in its own directory
  • Wait 24 hours and repeat

This will run on my main FATE machine, so run the script at a low priority so it doesn’t take time away from the more important build/test operations.

So that seems straightforward enough, especially drawing on the experience I have with FATE already (this is basically a simpler version of the main FATE script). For bonus points, the script should also put the freshest compiler into service automatically so I don’t have to be bothered with the tedious, error-prone process. This will be a bit trickier, but it should be possible:

  • Contact the primary FATE MySQL database @ fate.multimedia.cx and update the relevant records in the configuration table.
  • Signal the FATE script to start using the new compiler versions. In the present arrangement, while I have 2 scripts in order to leverage 2 CPU cores, 1 of the scripts is in charge of the 2 gcc-svn configuration for x86_32 and x86_64. The compiler path is specified in a Python configuration file. I may have to modify FATE a little bit so that the script checks some independent file for the compiler location. Then, this new script can update that location.

There is also the matter of storing so many old gcc-svn compiler versions. I’ll likely clean that up manually at various intervals but its not terribly pressing; the main FATE build/test machine inherited a 400 GB hard drive when it went into service (which, you’ll be pleased to hear, is partially used in performing backup duty for the various multimedia.cx domains).

Latest Compiler News

I’ve been doing compiler stuff tonight:

  • Thanks to Carl Eugen Hoyos as well as my compiler contact inside Intel for advising me on how to procure icc version 11.0.083 (vs. .081 previously) along with an unlimited, non-commercial compiler license. Looks like I won’t have to worry about the 31-day limit now (though there might still be a problem with the Mac OS X version). Further, the 32-bit compiler runs from the 64-bit kernel prompt (I am trying to move away from the 32-bit chroot setup and am meeting with some success). Both 32- and 64-bit versions are now in FATE.
  • After a brief respite following the release of gcc 4.4.0, I have updated the gcc SVN snapshots and reinstated the configurations tracking the FFmpeg build status on experimental gcc 4.5 builds. I’m especially proud, though, that I managed to build one C compiler binary that runs under 64-bit Linux but can build both 32- and 64-bit binaries.
  • A lot of people wonder about this, so I wanted to make it known that I have been briefed on how to use LLVM, the rising star of compiler technology. Thus far, I have not been able to create a build that compiles FFmpeg. I hear that they’re working on it.

New performance smackdowns to come, at least for those that can currently build FFmpeg (the current rev of gcc 4.5-SVN, rev 147090, isn’t doing so well– failing across platforms).

Performance Smackdown: The Latest in 64-bit From GCC and Intel

Since gcc 4.4.0 has been formally released, it’s time to re-run the compiler output benchmarks. Further, I finally sat down and put my mind toward getting the latest Intel C compiler installed and operational. I met with limited success. I haven’t been able to get the 32-bit compiler working. After the tedious rigmarole of getting version 11.0.081 installed, I launched the program without any parameters:

$ /opt/intel/Compiler/11.0/081/bin/ia32/icc
Segmentation fault

Grrrrr… why do I even bother? Fortunately, the intel64 (x86_64) compiler is operational. At the same time I was grabbing the Linux version, I noticed that there is a Mac OS X version, though it is somewhat down-rev at 11.0.059. I still downloaded that and tried it out. I was able to get it to build 32-bit binaries but not 64-bit.

So the upshot, FATE-wise, is that I have put 11.0.081/Linux/x86_64 and 11.0.059/Mac OS X/x86_32 into the system for continuous building and testing. At the time of this writing, they’re not doing so well. Lots of H.264 tests fail. The regressions pass for the most part, though.

But I stubbornly proceeded with the output benchmarks anyway. This is how the compilers are performing, per my usual method (best time out of 2 runs on the same, long, HD file; no hand-crafted ASM optimizations enabled):


64-bit compiler output performance chart, round 2

The gcc versions demonstrate similar performance to the first round of 64-bit tests. As for the icc 64-bit results, well, I don’t think I need to interpret that for you. I will tell you that I first ran it with no special options. Then I ran it with “–cpu=core2” which improved its run time by about 3 seconds. The gcc configurations used no special options.

However, there is a deeper issue. As indicated by the FATE tests, icc is incorrectly decoding H.264 video. Thanks to the 10-second validation files generated during the benchmarks, I am able to see that, what should look like this (from gcc 4.4.0):


64-bit validation file, generated by gcc 4.4.0

turns out like this (icc 11.0.081):


64-bit validation file, generated by icc 11.0.081

This makes me wonder what is so special about the FFmpeg H.264 decoder that icc has so much trouble digesting it. Is the code especially tricky? Or does it have a lot of tight loops that icc sees as opportunities for (mistaken) vectorization?

Another issue that concerns me regarding this latest series of Intel C compilers: I only have an evaluation license for 31 days. I’m not sure what happens after that. Presumably, I don’t get to use the compiler anymore. However, Intel seems to rev their compiler so often that I wonder if each minor update comes with a 31-day evaluation license.

See Also: