People offered a lot of constructive advice about my recent systematic profiling idea. As in many engineering situations, there’s a strong desire to get things correct at the start while at the same time, some hard decisions need to be made or else the idea will never get off the ground.

Code Coverage
A hot topic in the comments of the last post dealt with my selection of samples for the profiling project. It seems that the Big Buck Bunny encodes use a very sparse selection of features, at least when it comes to the H.264 files. The consensus seems to be that, to do this profiling project “right”, I should select samples that exercise as many decoder features as possible.

I’m not entirely sure I agree with this position. Code coverage is certainly an important part of testing that should receive even more consideration as FATE expands its general test coverage. But for the sake of argument, how would I go about encoding samples for maximum H.264 code coverage, or at least samples that exercise a wider set of features than the much-derided Apple encoder is known to support?

At least this experiment has introduced me to the concept of code coverage tools. Right now I’m trying to figure out how to make the GNU code coverage (gcov) tool work. It’s a bumpy ride.

Memory Usage
I think this project would also be a good opportunity to profile memory usage as well as CPU usage. Obvious question: How to do that? I see that on Linux, /proc/<pid>/status contains a field called VmPeak which is supposed to advertise the maximum amount of memory that the process has allocated. This might be useful if I can keep the process from dying after it has completed so that the parent process can read its status file one last time. Otherwise, I suppose the parent script can periodically poll the file and track the largest value seen. Since this is testing long running processes and I think that, ideally, a lot of necessary memory will be allocated up front, this approach might work. However, if my early FATE memories are correct, the child process is likely to hang around as a zombie until the final status poll(). Thus, check the status file before the poll.

Unless someone has a better idea.

Thanks to all of the advice in the comments of the last post about filling in gaps in the FATE test coverage, I have staged 11 new FATE test specs:

• aac-demux
• ea-mad-pcm-planar
• nc-demux
• pcm_dvd
• sp5x
• sunraster-1bit-raw, sunraster-1bit-rle, sunraster-8bit-raw, sunraster-8bit-rle, sunraster-24bit-raw, sunraster-24bit-rle

Regarding that group of 6 Sun raster files, it’s interesting to note that the 24-bit raw Sun raster file sample is smaller than the 24-bit RLE version.

I encountered a few problems with the suggestions from the last post. Among them:

• ami_stuff came through with a sample of a Fibonacci-encoded 8svx file. Unfortunately, it’s attached to a bug report because it’s not presently working. Test not staged.
• I downloaded the free Command & Conquer games from EA and looked into Tiberian Sun specifically. It looks like all the game resources are wrapped up into .MIX files. Not a big deal– I wrote a program years ago to take these apart. Unfortunately, the files are in a different MIX file format, apparently. So I’m still stuck on trying to get the audio samples I need.
• Carl Eugen pointed to a sample of Blu-Ray PCM (mpegts+h264+++trunc_read_packet_loop.m2ts). Thing is, the file has 9 streams; the pcm_bluray stream is right in the middle. I still don’t know how to tell FFmpeg to select that stream.
• On the subject of files that have more than 1 audio and 1 video stream, most of these samples with subtitles have the same problem as encountered with the last item– I don’t know how to tell FFmpeg to process the subtitle stream. In fact, the sample from the previous item also has 4 pgssub streams. How do I select one? And will I be able to cleanly mux a subtitle stream into the framecrc format?

I think FFmpeg’s -map option may hold the key. But I’m a little too tired and annoyed to read the source code which I’m certain is the only true documentation for how it works.

In my last post regarding recently-staged FATE tests, a number of Amiga sentimentalists expressed willingness to help me track down multimedia formats that were prevalent on that platform. To that end, I ask: Where do I find Fibonacci-encoded 8svx files? 8svx files can contain several audio codecs, but I have been unable to find ones with the Fibonacci format.

While we’re on the subject, I may as well put out a general call for samples that have eluded me:

• Fibonacci-encoded 8svx samples, as mentioned above
• ASS/SSA samples; plus, is there any good way to test ASS/SSA subtitles using ‘ffmpeg’?
• ADTS AAC: How do I generate that? I thought faac was supposed to help me with that but I couldn’t seem to get ADTS out of it.
• raw Ingenient MJPEG
• I know how to generate MPC (vs. MPC8 files, which I have already covered); the demuxer just doesn’t seem to work correctly right now.
• There are a number of formats like NC camera feed format, rtsp, and sdp that I suspect are impossible to test from disk rather than network.
• TXD: I think this is a raw format and that I have to supply parameters from the command line to decode it properly. I think these are valid TXD files but I don’t know their resolution (or, indeed, if they’re single images since TXD is supposed to be a texture dictionary).
• pcm_bluray and pcm_dvd: any VOBs in the archive with these data types?
• pcm_s16le_planar: Based on my code excavation, this is used in certain EA chunked formats, such as in NBA Live 2003 according to our wiki page on EA formats. We lack samples in the archive for that game. However, this reminds me that I really should modify the FILM/CPK demuxer so that it outputs planar audio instead of interleaving the audio in the demuxer (maybe someone else wants to get on top of that, if they’re looking for an easy task).
• pgssub, xsub: again, where are samples and how do I test subtitle formats?
• Sunplus JPEG (SP5X)
• Sun Rasterfile image
• Westwood Audio (SND1)

There are plenty of formats not covered yet according to the FATE test coverage page. For formats which have both an encoder and decoder in FFmpeg, I plan to have a better system in place in the next FATE version for testing those (which will also obviate the need for the {MAKETEST} test spec). Then there are the non-bitexact formats that require more advanced testing features which are in development.

Meanwhile, I learned that MPEG-4 ALS actually does have a formal conformance suite available (you can usually count on that for MPEG standards; take that, Xiph). So I will be disabling the current ad-hoc test spec and have staged 6 of the conformance vectors known to be correct (based on features that have been implemented thus far): 00, 01, 02, 03, 04, and 05. Further, 2 more new specs: iff-byterun1 and frwu are ready to go.