Category Archives: Python

Gymnastics Routine


Pommel horse

You would not believe how many numerical gymnastics I have to perform in order to test these MPEG-1 audio conformance vectors. It seems straightforward enough– a conformance vector, at least for layers 1 and 2, consists of a .MPG file and a .PCM file. The MPG file is supposed to contain an encoded MPEG audio stream while the PCM file has the output after the corresponding MPG file has been run through the official reference decoder. The root mean square (RMS) of the difference between that reference PCM file and, say, the output of the FFmpeg decoder needs to be less than 1 / (32768 * sqrt(12)). So what’s the big deal?

Continue reading →

Science Into Engineering

I modified my distributed RPC test staging utility to implement my imprecise audio testing idea. This is the output under typical conditions:

There was 1 unique stdout blob collected
all successful configurations agreed on this stdout blob:
pass

So, it worked. Yeah, I’m surprised too. That result means that all the configurations (20 total) produce an audio waveform in which no individual PCM sample deviates from the reference wave by more than 1. Since I had to choose some configuration to generate the reference sample, I used Linux / x86_32 / gcc 2.95.3.

BTW, this is the general Python algorithm I am using to compare the waves. It takes a full minute, give or take a second, to compare 2 33MB samples:

I replaced abs() with a branch to check if the diff is < -1 or > 1, but that didn’t improve speed measurably. I think the constant unpacking might have something to do with it. Better solutions welcome. (By comparison, performing a comparison using ‘cmp’ of 2 identical files that have the same size as the test above, living on a network share, takes less than 2 seconds.)

For a 10-second sample of a .m4a stereo AAC file (882,000 samples), these are the number of PCM samples that deviated by 1 (first number), and by more than 1 (second number). You will notice that no samples deviated by more than 1, which was my hypothesis at the start, and the basis on which I devised this plan:

Mac OS X / PPC / gcc 4.0.1
432691, 0

Linux / x86_32 / icc
238, 0

Linux / x86_32 / gcc 2.95.3
0, 0

Linux / PPC / gcc 4.0.4
Linux / PPC / gcc 4.1.2
Linux / PPC / gcc 4.2.4
Linux / PPC / gcc 4.3.2
Linux / PPC / gcc svn
432701, 0

Linux / x86_64 / gcc 4.0.4
Linux / x86_64 / gcc 4.1.2
Linux / x86_64 / gcc 4.2.4
Linux / x86_64 / gcc 4.3.2
Linux / x86_64 / gcc svn
248, 0

Linux / x86_32 / gcc 3.4.6
Linux / x86_32 / gcc 4.0.4
Linux / x86_32 / gcc 4.1.2
Linux / x86_32 / gcc 4.2.4
Linux / x86_32 / gcc 4.3.2
Linux / x86_32 / gcc svn
237, 0

Mac OS X / x86_64 / gcc 4.0.1
244, 0

I have thrown RealAudio Cooker and 28.8 samples at this, and both work. I am still testing this against some more audio samples to ensure that this idea holds water.

Parsing In Python

I wanted to see if the video frames inside these newly discovered ACDV-AVI files were just regular JPEG frames stuffed inside an AVI file. JPEG is a picky matter and many companies have derived their own custom bastardizations of the format. So I just wanted to separate out the data frames into individual JPEG files and see if they could be decoded with other picture viewers. Maybe FFmpeg can already do it using the right combination of command line options. Or maybe it’s trivial to hook up the ‘ACDV’ FourCC to the JPEG decoder in the source code. What can I say? FFmpeg intimidates me just as much as it does any of you mere mortals.

Plus, I’m getting a big kick out of writing little tools in Python. For a long time, I had a fear of processing binary data in very high level languages like Perl, believing that they should be left to text processing tasks. This needn’t be the case. pack() and unpack() make binary data manipulation quite simple in Perl and Python. Here’s a naive utility that loads an AVI file in one go, digs through it until it finds a video frame marker (either ’00dc’ or — and I have never seen this marker before — ’00AC’) and writes the frame to its own file.

acdv.py:

BTW, the experiment revealed that, indeed, the ACDV video frames can each stand alone as separate JPEG files.

Processing Those Crashers

I know my solutions to certain ad-hoc problems might seem a bit excessive. But I think I was able to defend my methods pretty well in my previous post, though I do appreciate the opportunity to learn alternate ways to approach the same real-world problems. But if you thought my methods for downloading multiple files were overkill, just wait until you see my solution for processing a long list of files just to learn — yes or no — which ones crash FFmpeg.

So we got this lengthy list of files from Picsearch that crash FFmpeg, or were known to do so circa mid-2007. Now that I have downloaded as many as are still accessible (about 4400), we need to know which files still crash or otherwise exit FFmpeg with a non-zero return code. You’ll be happy to know that I at least know enough shell scripting to pull off a naive solution for this: Continue reading →