Tag Archives: continuous integration

Investigating Alternate Continuous Integration Software

Ever since I figured out that FATE falls into the category of continuous integration (CI) software, I have been evaluating alternate, established, mature software packages that might be able to supplant FATE for managing FFmpeg automated building and testing. I don’t think it’s fair to say that I have been disappointed with the offerings I have found out there; it’s just that many other software packages have apparently approached this problem from a slightly different angle, with a slightly different set of assumptions, and slightly different resources on which to deploy a solution.

For starters, when I read the overviews for a number of these packages, I am never quite sure if they actually do what FATE already does. Many packages brag about their amazing abilities to perform continuous builds on many platforms. I might be missing something crucial here, but that fact alone doesn’t impress me. From my perspective, and in my experience, automating the build among multiple independent machines is the easy part. The difficult part is managing tests and aggregating the results back to a central location.

Aside: Sometimes I back up and wonder: Why aren’t our tests integrated into the build process? Oh yeah, they are: ‘make test’. But why can’t that general test be made to cover all the functionality? Because the ‘make test’ regression suite depends on the existence of both an encoder and a decoder for a particular format. FFmpeg covers a large amount of formats that it can only decode, not encode (and no one is going to step up to write even naive corresponding encoders for these formats in the near future, nor would I argue that they should). Therefore, it becomes necessary to test asymmetric decoder (and demuxer) functionality using individual tests as I designed FATE to do.

I designed FATE such that a client program would run on diverse platforms, constantly check for and rebuild new FFmpeg code, run a battery of tests, and log the results of the build and tests (including status, stdout, stderr, and performance stats) back to a central server. I wanted the client to be written in Python and use only the (expansive) standard Python library, if possible. More stringent were the server requirements: I needed something that operated via HTTP backed by PHP or Perl — because that’s what my web host provides, at least if I want to talk to MySQL (Ruby is also an option; so is Python as long as the CGI script doesn’t need to talk to MySQL).

“But you could host the central server at home on your own broadband and use whatever configuration you want.” Actually, administering things like web and email services is something I would rather pay a few dollars per month for someone else to do on my behalf. I am quite adamant about this. I feel the same way about laundry.

“If FATE ain’t broke, why fix it?” Well, it is sort of broken, or at least incomplete. Many of these packages already boast features like email notifications and even IRC notifications, in addition to better web interfaces– features that I would like to implement when time permits. And at the very least, surveying other CI software might give me ideas about how to improve FATE.

Can existing CI packages solve my problems? Continue reading

Asking The Right Question

Considering the amount of time and effort I put into developing the entire FATE system, you might be surprised to learn that I would not be at all averse to replacing FATE wholesale with something that worked better. I did research at the outset to see what kind of software systems were out there that would suit our needs and solve all of the problems that I had in mind. But I couldn’t find much useful stuff. To be honest, I wasn’t entirely sure what I was looking for.

In order to find the correct answer, though, it helps immensely to know the right question. Through a series of coincidences, I wound up at the Wikipedia page for continuous integration and realized that this is the category of software that FATE falls into. The Wikipedia page lists many systems that are used along the same lines as FATE.

BuildBot is an interesting one and a system that I think I have seen before. Python-based, good. Example report pages are well-organized, but not as concise as I think they could be (but perhaps it’s configurable). However, I tend to think that there are few continuous integration systems that meet a particular requirement I have, namely that the master server needs to be able to run on PHP since that’s what my web provider offers (Python-CGI, too, as long as I don’t need to talk to a MySQL database).