Breaking Eggs And Making Omelettes

Topics On Multimedia Technology and Reverse Engineering


Archives:

Belief In The Compiler

February 21st, 2008 by Multimedia Mike

If you keep up with FATE as obsessively as I do, you may have noticed that I got Intel’s C compiler (icc) into the build farm. It was a struggle, but I finally made it happen. The compiler is distributed as an RPM. but the x86_32 build machine is Ubuntu. I googled and found a number of blog posts describing how to install it on Ubuntu. I went the route of using the alien program to convert the RPM to a DEB, installing it, manually modifying the ‘icc’ shell script to point to the correct INSTALLDIR and updating the ld configuration to point to the right libraries. Finally, I installed the free-for-non-commerial-use license file in one of the many acceptable locations and I was off.

When I first started fighting with icc about a month ago, the compiler was on version 10.1.008. It is now on 10.1.012, indicating that I may need to update it almost as frequently as the the SVN version of gcc used on the farm.

I also tried to get the x86_64 version of the compiler running on the appropriate build machine. When I try to run ‘icc’, I get one of the most annoying and confusing UNIX errors known to exist:

  -bash: /opt/intel/cce/10.1.012/bin/iccbin: No such file or directory

Even though I can see that file quite clearly exists.

But at least icc is running for x86_32, and FFmpeg is compiling fine and running the same series of tests as all the gcc versions. Personally, I have never put a lot of stock in the optimizing prowess of proprietary compilers. I have seen a few too many that need to have their optimizers disabled because they are so obviously buggy. However, icc demonstrates some clear speedups over gcc based on FATE testing. If you open a build record page for an icc run in one window or tab, and then open a build record page for a gcc run in another, you can see that the icc-built binary generally runs faster. This is particularly notable on longer tests.

This exercise also reminds me that the SVN versions of gcc build slow binaries, at least on x86_32. I am wondering if this has to do with the way I am building the compiler, or if 4.3 will actually build substantially slower binaries?

And yes, I plan one day to deploy an easier way of comparing build performance over time, and for various platforms.

Posted in FATE Server | 6 Comments »

6 Responses

  1. astrange Says:

    Run it under strace and see where it actually fails.
    I wish there was a free icc for OS X; oprofile never works when I run it, so I can’t get any useful optimization work done in Linux.

    Someday you might want to add LLVM: http://llvm.org/
    I think it miscompiles most of the inline asm at the moment, though.

  2. Multimedia Mike Says:

    The strace run was rather short and ultimately fruitless. However, it did lead me to wonder whether this is a 64-bit binary. According to ‘file’, it’s a 32-bit executable. Attempting to run ‘ld’ against it reports that, “i386 architecture of input file `/opt/intel/cce/10.1.012/bin/iccbin’ is incompatible with i386:x86-64 output”.

    It’s possible that there is a different version I was supposed to download.

  3. Reimar Says:

    I know for sure that there is an ICC version that is a 64 bit binary (I use it and my gentoo can’t run anything that needs 32 bit libraries). Actually I think even the compiler to 32 bit exists as a 64 bit binary.
    And I personally install ICC with the shell script and as normal user. I have seen binary RPM/DEBs far too often mess up a system to run them as root anymore (even though Intel is one of the companies I’d trust to properly test even their linux packages).

  4. Anonymous Says:

    so, i guess Sun C, MIPS Pro and Microsoft C are next?
    — pete

  5. Multimedia Mike Says:

    I do know someone who wants to get this system running on some Solaris boxes (presumably both x86 and Sparc). I need to refactor some code and network protocols before that can happen, though.

    I suppose its up to me to get something to work for MS compilers. I don’t know anyone with access to MIPS hardware and a proprietary compiler for it.

  6. Reimar Says:

    Sun C needs a SPARC machine, going by my tests it has no chance to work on x86, even with all asm disabled due to an amazing amount of compiler bugs.
    Since Microsoft has no compilers I know of that are even remotely C-compliant (in particular, near and far were still keywords when I last checked) I can not see how they could be used in FATE.
    I do not know it, but I would assume that MIPS Pro could also run as cross-compiler, though that still leaves the problems of
    1) getting a license
    2) a compiler that does not crash when used on the FFmpeg code for the first time would be a real miracle.