Category Archives: General

Progress On Those Crashers

Last December, I set about on the task of downloading and testing a huge number of files that were known, at one point, the crash FFmpeg. I devised a system for automatically running the files and determining whether they still crash in FFmpeg. Quite a few of them did. Then, I sort of let the project sit.

I got around to running a new round of tests with the utility I created in December and compared the results with those of 4 months ago. Today’s test was conducted with FFmpeg SVN-r18707 built with “gcc: 4.0.1 (Apple Inc. build 5484)”, 32-bit version, and run on Mac OS X.

Result December 8, 2008 April 27, 2009
Success 2148 2781
FFmpeg error 1333 1389
SIGABRT 6 6
SIGFPE 376 1
SIGKILL (timeouts) 16 17
SIGBUS 7 97
SIGSEGV 529 123

Great progress, especially on those floating point exceptions. I’m pretty sure nearly all of those were attributable to one or a few problems in the Real demuxer that have since been addressed. The only remaining problem in the FPE category is an AVI file.

The timeout category represents the number of files that ran longer than a minute (need to keep the process moving). The “FFmpeg error” category (return code 1) is on the rise. I surmise that’s because FFmpeg is getting better at rejecting errant files vs. crashing on them. I should really formulate a query that reveals which files’ status changed, and how, between runs.

A big reason I sat on this project for so long is that I didn’t know how to proceed. Should I start testing the problem files manually, collect stack traces, and flood the FFmpeg issue tracker with hundreds of new reports? I don’t want to deal with that kind of manual labor and I don’t think my co-devs want to deal with that volume of (possibly redundant) bug traffic.

Since December, I have developed another idea: Automatically running the problem files through gdb and looking for patterns. For example, I manually checked those 6 crashers that threw SIGABRT (the same 6 files from each run, BTW, and all ASF files). They all seem to fail as follows:

Program received signal SIGABRT, Aborted.
0x96dbbe42 in __kill ()
(gdb) bt
#0  0x96dbbe42 in __kill ()
#1  0x96dbbe34 in kill$UNIX2003 ()
#2  0x96e2e23a in raise ()
#3  0x96e3a679 in abort ()
#4  0x96e2f3db in __assert_rtn ()
#5  0x00026529 in ff_asf_parse_packet (s=0x1002600, pb=0xa00200, 
pkt=0xbfffe954) at /Users/melanson/ffmpeg/ffmpeg-main/libavformat/asfdec.c:709

It would be nice to create a script that identifies that all 6 of those files suffer from the same, or similar problem and group those files together in a report. I am not sure if gdb offers non-interactive options that are conducive to this situation. I know it has a -batch mode, but I’m not really sure what that’s for. If need be, I can always create a Python script that opens gdb in interactive mode and has a stdin/stdout conversation with it.

See Also:

Of Filesystems and Codecs

I have been hanging out at the Linux Foundation Collaboration Summit. One theme I have heard tossed around is the matter of filesystems– ongoing filesystem research, the need to upgrade standard filesystems in Linux, etc. I admit that I don’t spend a lot of time thinking about filesystems (except when I’m writing FUSE drivers for filesystems that lack wide appeal). The filesystem is something that’s just “there” and should just work. Indeed, I have never had a major problem with any filesystem I have used while it is still considered modern. It is only when the next generation comes along that I understand the faults in the previous generation (journaled filesystems helped me understand that extensive integrity checking at boot time doesn’t have to be necessary; anything beyond FAT16 helped me understand that 8.3 filenames didn’t have to be the standard).

But there is a category of obsessed individuals who spend a lot of time thinking about filesystems and measuring what they’re doing and figuring out how they could be doing things better. And it’s a good thing that we have these people around, even though most of us largely view filesystems as a transparent cog in the machine of daily computing.

This got me to thinking about how it’s probably very likely that most computer users view multimedia codecs the same way that I view filesystems. An AVI file might contain Cinepak or MPEG-4 part 2 video, or any of 100+ video codecs. Most users don’t have a reason to care about the difference. This may help to explain why some people (not particularly well-versed in multimedia technology) take it for granted that Theora could easily replace H.264 in all applications where the latter is in use today.

They’re both video codecs, right?

Performance Smackdown, Now With 64-bit

Another in my continuing series of compiler performance reports– that is, the performance of straight C code when compiled by assorted compilers. Pursuant to round 3, I downloaded the long, free, hi-def H.264/AAC movie to profile, as suggested by Reimar and profiled that. It takes 11-15 minutes to decode the entire thing on my 2.13 GHz Core 2. No matter; my machine is patient, and here are the results:


icc vs gcc performance chart when running FFmpeg, round 4

“gcc-svn” is gcc 4.4.0-svn, revision 143046, built on 2009-01-03, same as before.

All validations passed. Further, I used “march=pentium4” as suggested by Flameeyes, on compilers that supported the option but not “march=core2” (gcc 3.4.6, 4.0.4, 4.1.2, and 4.2.4). I think that improved performance for those, but I won’t know for sure unless I run with the original MPEG-4 part 2/MP3 movie from the previous tests.

I also took this opportunity to see how native 64-bit builds performed on the same machine. I hope one day to get Intel’s 64-bit compiler working so it can be included in the competition:


Profiling 64-bit code using FFmpeg

For this test, I didn’t specify any compiler optimizations from the command line. Let me know if that should change for the next round. “gcc-svn” is a little more up to date at gcc 4.4.0-svn, revision 144720, built on 2009-03-08.

Lingering TODO: Investigate if Acovea can help in this process.

See Also:

All Web Hosts Suck

The last few weeks have taught me that all web hosts suck. Ordinarily, I don’t feel it necessary to write a blog post railing and ranting against a company that annoys me; I do my best to let these things go quickly. However, according to the “sucky research method”, WebFaction has thus far managed to escape any negative criticisms. This wouldn’t be such a problem except that they’re a tad smug about it. Further, I think I am having trouble achieving emotional closure and catharsis on this matter since I find no similarly suffering souls out there with whom to commiserate. So it is with a heavy heart that I feel compelled to type out a petty, petulant “WebFaction sucks” post.

Who knows? Maybe I’m just the unluckiest customer a web host has ever had. Continue reading →