Tag Archives: Python

Constantly Compiling Compilers

At the outset of this FATE journey, I pondered how frequently I ought to update gcc from SVN and put new experimental gcc versions into service. At that point, I had some logistical difficulties in upgrading compilers due to my computer setup which made it a serious chore. But it’s much easier to do now– at least for x86_32 and x86_64, I know how to build one compiler that can build both binaries. Further, the C compiler only takes about 45 minutes to compile on my 2.13 GHz Core 2 Duo, leveraging both cores, and barring any other CPU activity.

So, if you know me, you should know that right about now I’m thinking… automated script! Here’s the pitch:

  • Write a new Python script that runs continuously which performs the following steps:
  • ‘svn update’ of my local copy of gcc-svn
  • If the SVN number is higher than the latest build, automatically build a new version and install it in its own directory
  • Wait 24 hours and repeat

This will run on my main FATE machine, so run the script at a low priority so it doesn’t take time away from the more important build/test operations.

So that seems straightforward enough, especially drawing on the experience I have with FATE already (this is basically a simpler version of the main FATE script). For bonus points, the script should also put the freshest compiler into service automatically so I don’t have to be bothered with the tedious, error-prone process. This will be a bit trickier, but it should be possible:

  • Contact the primary FATE MySQL database @ fate.multimedia.cx and update the relevant records in the configuration table.
  • Signal the FATE script to start using the new compiler versions. In the present arrangement, while I have 2 scripts in order to leverage 2 CPU cores, 1 of the scripts is in charge of the 2 gcc-svn configuration for x86_32 and x86_64. The compiler path is specified in a Python configuration file. I may have to modify FATE a little bit so that the script checks some independent file for the compiler location. Then, this new script can update that location.

There is also the matter of storing so many old gcc-svn compiler versions. I’ll likely clean that up manually at various intervals but its not terribly pressing; the main FATE build/test machine inherited a 400 GB hard drive when it went into service (which, you’ll be pleased to hear, is partially used in performing backup duty for the various multimedia.cx domains).

icc vs. gcc Smackdown, Round 3

How did I become the benchmark peon? Oh right, I actually dared to put forth some solid benchmarks and called for suggestions for possible improvements to the benchmark methodology. This is what I get.

Doing these benchmarks per all the suggestions I have received is time-consuming and error-prone. But if you know anything about me by now, you should know that I like automating time-consuming and error-prone tasks. This problem is looking more and more like a nail, so allow me to apply my new favorite hammer: Python!

Here’s the pitch: Write a Python script that iterates through a sequence of compiler configurations, each with its own path and unique cflags, and compiles FFmpeg. For each resulting build, decode a long movie twice, tracking the execution time in milliseconds. Also, for good measure, follow Reimar’s advice and validate that the builds are doing the right thing. To this end, transcode the first 10 seconds of the movie to a separate, unique file for later inspection. After each iteration, write the results to a CSV file for graphing.

And here’s the graph:


icc vs. gcc smackdown, round 3

Look at that! gcc 4.3.2 still isn’t a contender but gcc 4.4-svn is putting up a fight.

Here are the precise details of this run:

  • Movie file is the same as before: 104-minute AVI; ISO MPEG-4 part 2 video (a.k.a. DivX/XviD) at 512×224, 24 fps; 32 kbps, 48 kHz MP3
  • This experiment includes gcc 4.4.0-svn, revision 143046, built on 2009-01-03 (I’m a bit behind)
  • All validations passed
  • Machine is a Core 2 Duo, 2.13 GHz
  • All 8 configurations are compiled with –disable-amd3dnow –disable-amd3dnowext –disable-mmx –disable-mmx2 –disable-sse –disable-ssse3 –disable-yasm
  • icc configuration compiled with –cpu=core2 –parallel
  • gcc 4.3.2 and 4.4.0-svn configurations compiled with -march=core2 -mtune=core2
  • all other gcc versions compiled with no special options

See Also:

What’s in store for round 4? It sure would be nice to get icc 11.0 series working on my machine for once to see if it can do any better. And since I have the benchmark framework, it would be nice to stuff LLVM in there to see how it stacks up. I would also like to see how the various builds perform when decoding H.264/AAC. The problem with that is the tremendous memory leak that slows execution to a crawl during a lengthy transcode. Of course I would be willing to entertain any suggestions you have for compiler options in the next round.

Better yet, perhaps you would like to try out the framework yourself. As is my custom, I like to publish my ad-hoc Python scripts here on my blog or else I might never be able to find them again.

Continue reading →

Gymnastics Routine


Pommel horse

You would not believe how many numerical gymnastics I have to perform in order to test these MPEG-1 audio conformance vectors. It seems straightforward enough– a conformance vector, at least for layers 1 and 2, consists of a .MPG file and a .PCM file. The MPG file is supposed to contain an encoded MPEG audio stream while the PCM file has the output after the corresponding MPG file has been run through the official reference decoder. The root mean square (RMS) of the difference between that reference PCM file and, say, the output of the FFmpeg decoder needs to be less than 1 / (32768 * sqrt(12)). So what’s the big deal?

Continue reading →

Implementing The RPC Idea

About that RPC-based distributed test staging idea I brainstormed yesterday, I’ll have you know that I successfully implemented the thing today. I used the fraps-v4 test spec for verification because it is known to work correctly right now, and because it only has 7 lines of stdout text. This is what the script looks like in action:

$ ./rpc-dist-test.py "FFMPEG -i 
  SAMPLES_PATH/fraps/WoW_2006-11-03_14-58-17-19-nosound-partial.avi 
  -f framecrc -" 
asking for results from 12 configurations...
testing config 0: Linux / x86_64 / gcc 4.0.4
testing config 1: Linux / x86_64 / gcc 4.1.2
testing config 2: Linux / x86_64 / gcc 4.2.4
testing config 3: Linux / x86_64 / gcc 4.3.2
testing config 4: Linux / x86_64 / gcc svn
testing config 5: Linux / PPC / gcc 4.0.4
testing config 6: Linux / PPC / gcc 4.1.2
testing config 7: Linux / PPC / gcc 4.2.4
testing config 8: Linux / PPC / gcc 4.3.2
testing config 9: Linux / PPC / gcc svn
testing config 10: Mac OS X / x86_32 / gcc 4.0.1
testing config 11: Mac OS X / x86_64 / gcc 4.0.1

1 configuration(s) failed
  configuration Mac OS X / x86_32 / gcc 4.0.1 returned status 133

There was 1 unique stdout blob collected
all successful configurations agreed on this stdout blob:
0, 0, 491520, 0x68ff12c0
0, 3000, 491520, 0x22d36f0d
0, 6000, 491520, 0xce6f877d
0, 9000, 491520, 0x85d6744c
0, 12000, 491520, 0x1aa85794
0, 15000, 491520, 0x528d1274
0, 18000, 491520, 0x357ec61c

A few notes about the foregoing: Continue reading →