{"id":502,"date":"2008-01-14T16:30:21","date_gmt":"2008-01-15T00:30:21","guid":{"rendered":"http:\/\/multimedia.cx\/eggs\/the-server-of-fate\/"},"modified":"2009-04-03T09:34:16","modified_gmt":"2009-04-03T16:34:16","slug":"the-server-of-fate","status":"publish","type":"post","link":"https:\/\/multimedia.cx\/eggs\/the-server-of-fate\/","title":{"rendered":"The Server of Fate"},"content":{"rendered":"

Pursuant to the last post’s naming contest<\/a>, SvdB had a novel entry of “FFmpeg Make \u2018n\u2019 Break”. However, Kostya’s entry of FATE was destined for victory due to its sheer simplicity. And so it comes to pass:<\/p>\n

FATE – FFmpeg Automated Testing Environment<\/strong><\/a><\/p>\n

Some may have observed that there still are not very many tests yet. I’m being slow and deliberate with these, at least at the outset. My first impulse was to start manually adding tests to validate a bunch of the fringe formats that I’m most familiar with (since I implemented them), as I have done with this test for the FILM system<\/a>. However, the guru<\/a> recommended that I put the H.264 conformance suite<\/a> to the test.<\/p>\n

The base directory has 136 samples. Yeah, I’m leaning towards automated tool<\/strong> on this one.<\/em><\/p>\n

This FATE project is prompting me to craft a variety of special tools to both make my life easier and ensure fewer errors.I could just make a tool to dump all the samples into the database, pass or fail, and let the test failure count tell the story. However, that might not be useful in the same way that it’s not useful to have hundreds of warnings in a compilation — it distracts from real problems (i.e., we know that 100 or so tests are supposed to fail and we don’t notice when a formerly working test just broke).<\/p>\n

I also figured out that it’s not so straightforward to dump all the tests in at once, at least not with correct results. Each archive has, at a minimum, a raw H.264-encoded file and the raw YUV file. A decode of the H.264 file is supposed to be bit exact when compared to the raw file. You can feed the raw YUV image into FFmpeg (and encode to the framecrc target for concise stdout text), but only if you know the file’s resolution. The samples usually have readme files included, and they usually mention the resolution, but I’m not going through that much trouble to pick it out. I’ve already worked out the regexps to figure out what the encoded, raw, and readme files can possibly be named.<\/p>\n

So my current plotted strategy works like this; for each .zip file in the conformance suite:<\/p>\n