I resisted adding ‘make test’ as an individual FATE test for a long time because it was too big, took too long to run, and because it’s an all or nothing proposition (i.e., if one test fails, the whole test is marked as a failure). Plus, I eventually want to break up each individual test in the ‘make test’ regression suite into individual FATE tests. But a few months ago, I relented — until I make the big test split — and entered test spec #128, which runs ‘make test > /dev/null 2>&1’. The redirection was necessary because ‘make test’ generates way more data than I care to track. It was suggested that I should try to do some shell magic to capture the final n lines of output in case of failure. This is a nice idea. But I couldn’t figure out how to do it due to the fact I can’t stand shell scripting.
I was recently trying to process a huge number of problematic multimedia samples using shell scripting commands. It didn’t work well. I resorted to a custom Python script which worked much better. This little episode reminded me of some other shell workarounds I deployed in FATE such as the {FILESIZE} and {MD5} custom strings. Instead of shell commands to obtain information for certain tests, I map these special commands to cleaner, more portable Python code. And it finally occurred to me to do the same for the full regression suite.
Say hello to the new test spec #128— {MAKETEST}. The FATE script recognizes this command and realizes that it should run ‘make test’, capture the stdout/stderr, clear both if the suite succeeds, or chop both to only include the last 30 lines if anything in the regression suite went bad. This will help developers study why the suite is failing on various systems.
At least until I finally develop a good plan for breaking the master regression suite into several hundred little tests.
Yay! Though I think PPC gcc svn is still beyond hope of fixing ;-)
Fully splitting the regression tests is on my list of things to do some time.
Well, you know, me and Luca work with shells scripts for ebuilds all day long you could have asked ;)
md5sum $filename | cut -d ‘ ‘ -f1
this will give you just the md5 of the file without the filename.
size -c ‘%s’ $filename
will give you the size in bytes on GNU userlands, I admit I don’t remember the equivalent using BSD stat command, but you can otherwise use the find command
find $filename -printf ‘%s\n’
And now tell me that Python is more readable than this ;) Okay I admit it’s less portable, though I dislike Python’s “readability” very much, Ruby forever :)
It’s funny that you should present those ‘md5sum’ and ‘size’ command lines– because that’s exactly what I used to do during the first pass at FATE. Each installation had a TOOLS directory that had symlinks to the proper command line tool. The thinking behind the symlinks was that, for example, md5sum is not available everywhere (it’s just ‘md5’ on Mac OS X). I think it’s a cleaner solution to have this functionality inside of the Python script, and it’s a very minor amount of code to compute an MD5 hash or obtain a file’s size (and it’s part of the standard lib to boot).
I misremembered OSX had md5sum rather than md5 (the two have indeed different syntax too), md5 is also present on BSD.
As I said, it’s less portable, probably lots less portable, and if you use Python for the rest of the framework it’s definitely better to use it, I just find Python a lot less readable :P