At long last, I have a reasonable — and much improved — autobuild/test infrastructure running for FFmpeg. Visit it at builds.multimedia.cx. It’s a bit sparse right now as it only covers builds for 2 platforms (x86_32 and PowerPC, both on Linux) and only a handful of tests, which are actually pseudo-tests, only logging the filesizes of the ‘ffmpeg’, ‘ffplay’, and ‘ffserver’ binaries. Also, you will notice that the testbed makes an effort to keep up with recent SVN builds of gcc.
I’m hopeful for this new infrastructure; the sky’s the proverbial limit. In the short term, I will be adding x86_64/Linux builds. Solaris/Sparc and Mac OS X builds might be on the way as well. My other high priority right now is to create an administrative web form that will enable me (and hopefully some co-admins) to add and edit test specifications easily. It’s not ergonomic to do this through the MySQL console. Here are the top test ideas so far:
- All of those ‘make test’ regression tests– break those up into individual tests
- Type ‘ffmpeg -formats’– set up tests for each and every one of those individual modules; use the libavformat/framecrcenc module here
- Fetch collections of conformance vectors for various MPEG standards and run through them
The comments section is open for suggestions. Be ambitious. However, here’s an algorithmic challenge if you’re up to it. Look at the stderr output from this build of gcc 2.95.3/x86_32 that failed. It would be useful to highlight the errors. How do I search the text and find the error so I can highlight them, and also insert HTML anchor links for easy skipping? The best I can think to do is searching for ‘***’ to indicate errors and search back somehow.
I suggest regexp – :: (not “warning:”)
or cat log |grep “:[0-9]*: ” | grep -v ” warning: “
second one above should work, modified:
grep -n ‘:[0-9]+:’ log | grep -v “warning: ”
if you’ve got tons of CPU time left over, how about graphing CPU/binary size benchmarks? You could even do with/without –arch=generic to see if the no-asm version ever gets closer to the normal one.
Here’s a Poor Man’s colorizer for several compilers:
http://cvs.berlios.de/cgi-bin/viewcvs.cgi/mpm/pmcolorcc/
It can easily be adapted for HTML output. It currently outputs ANSI color codes.
astrange: Indeed, graphs are in the works, probably periodically updated offline and cached.
I have simpler idea.
If build fails, run `make` again and collect its output. It would contain only the error and the warnings of the failed module.
Sometimes stuff like “imlib2.c:60: warning: No include path in which to find Imlib2.h” could be dead giveaway :)
@iive: Thanks for demonstrating that, no matter how much thought I put into an architecture, there will still be a possibility I didn’t consider. :-)
In the end, I’m hoping that this thing will facilitate the cleanup of many warnings so that coloring specific errors will be unnecessary (since errors should be the only things showing up in the stderr text).
To make pretty graphs, you may be interested in:
http://code.google.com/p/flot/
which uses http://jquery.com/
The person who pointed this out to me says it uses Mozilla Canvas via those libraries. The graphs he showed me looked pretty anyway. :)
Thanks for the pointer, Robert. I haven’t heard of the Mozilla Canvas before but it sounds awfully Firefox-specific. I’m hoping this will work on a few more browsers than that. At the outset, I will probably be doing periodically updated graphs using gd and perhaps other libraries that sit on top of gd.
Later on, I might eat my own company’s dog food and make some agent using Adobe Flex. But I have to remember my customers here.
Oh wait, I read a little further down and saw that the canvas is also available on Apple’s Safari browser. So I guess I could craft a canvas-based solution that would work for 1.7% of the total web surfing population.
if you read a little more carefully you will see it mentions that is uses a javascript library to emulate canvas on ie. so all is good.
Good to know, Mark. Anything to push the graphical processing to the client, rather than worry about it on the server. I look forward to getting pretty pictures working after I populate the database with many more tests and get more architectures building.