Constantly Compiling Compilers

At the outset of this FATE journey, I pondered how frequently I ought to update gcc from SVN and put new experimental gcc versions into service. At that point, I had some logistical difficulties in upgrading compilers due to my computer setup which made it a serious chore. But it’s much easier to do now– at least for x86_32 and x86_64, I know how to build one compiler that can build both binaries. Further, the C compiler only takes about 45 minutes to compile on my 2.13 GHz Core 2 Duo, leveraging both cores, and barring any other CPU activity.

So, if you know me, you should know that right about now I’m thinking… automated script! Here’s the pitch:

  • Write a new Python script that runs continuously which performs the following steps:
  • ‘svn update’ of my local copy of gcc-svn
  • If the SVN number is higher than the latest build, automatically build a new version and install it in its own directory
  • Wait 24 hours and repeat

This will run on my main FATE machine, so run the script at a low priority so it doesn’t take time away from the more important build/test operations.

So that seems straightforward enough, especially drawing on the experience I have with FATE already (this is basically a simpler version of the main FATE script). For bonus points, the script should also put the freshest compiler into service automatically so I don’t have to be bothered with the tedious, error-prone process. This will be a bit trickier, but it should be possible:

  • Contact the primary FATE MySQL database @ fate.multimedia.cx and update the relevant records in the configuration table.
  • Signal the FATE script to start using the new compiler versions. In the present arrangement, while I have 2 scripts in order to leverage 2 CPU cores, 1 of the scripts is in charge of the 2 gcc-svn configuration for x86_32 and x86_64. The compiler path is specified in a Python configuration file. I may have to modify FATE a little bit so that the script checks some independent file for the compiler location. Then, this new script can update that location.

There is also the matter of storing so many old gcc-svn compiler versions. I’ll likely clean that up manually at various intervals but its not terribly pressing; the main FATE build/test machine inherited a 400 GB hard drive when it went into service (which, you’ll be pleased to hear, is partially used in performing backup duty for the various multimedia.cx domains).

3 thoughts on “Constantly Compiling Compilers

  1. Boris

    Im a little concerned at the frequency you plan to update GCC, this is bordering on a compulsive-disorder. What do you plan to achieve between daily updates?

  2. Multimedia Mike Post author

    Exactly. We have seen some problems creep into different builds of gcc-svn and when the last build was from 2 months prior, that makes it difficult to even begin to diagnose what might have caused a problem. (However, it has been coming to light that some of the breakages might be due to tightening of gcc conventions, necessitating a loosening of the compiler options).

    I don’t think this is any more OCD than what FATE does. And as long as its automated, it’s no huge burden to a human.

Comments are closed.