Breaking Eggs And Making Omelettes

Topics On Multimedia Technology and Reverse Engineering


Archives:

Libsndfile Survey; CAFF

May 12th, 2008 by Multimedia Mike

I have been sitting on the results of this little experiment for a month now. I was in a bit more of a hurry during the qualification period for this year’s Summer of Code because I knew it would eventually yield a bumper crop of suitable qualification tasks. But that time has long passed.

The task at hand is to systematically survey the types of files that libsndfile can create, see what it can do that FFmpeg can’t, and make a plan to get those formats into FFmpeg starting with listing them as small tasks.

Libsndfile comes with several small example utilities (something can FFmpeg could probably use instead of, or to supplement, one big utility). One such program is sndfile-convert

  1. import commands
  2.  
  3. formats = (
  4.         "aif",
  5.         "wav",
  6.         "...many formats omitted...",
  7.         "xi"
  8. )
  9.  
  10. codecs = (
  11.      "pcms8",     # force the output to signed 8 bit pcm
  12.      "pcmu8",     # force the output to unsigned 8 bit pcm
  13.      "...many formats omitted...",
  14.      "dpcm16",    # force the output to 16-bit DPCM (XI only)
  15. )
  16.  
  17. for format in formats:
  18.     for codec in codecs:
  19.         command = "sndfile-convert -%s cow.wav samples/%s-%s.%s" % (codec, format, codec, format)
  20.         print command
  21.         (status, output) = commands.getstatusoutput(command)
  22.         if (status != 0):
  23.             print "  ** %s" % (output)

I had to modify sndfile-convert since not all of the supported formats were enumerated in its case-switch statements. The result of the above command was 120 unique format/codec combinations. I have uploaded all 120 cow moo samples (taken from the OpenOffice media suite, handy on my Eee PC when I was running this experiment).

How many of these 120 files can FFmpeg decode? I compiled FFmpeg with GSM 6.10 support and used the following command to print out FFmpeg return code and filename for each sample:

for file in `ls`; do
  ffmpeg -i $file -f wav - 1> /dev/null 2> /dev/null;
  echo $? $file; 
done

The count is: 26 supported, 94 unsupported. Time to get to work. You wouldn’t believe how many different ways there are to wrap raw PCM data with a basic file header for storage and transport.

Actually, closer inspection will reveal that FFmpeg is not necessarily ready to support all of these file formats since a number of them contain 24- or 32-bit integer PCM, or 32- or 64-bit floating point PCM; these are longstanding FFmpeg TODO items.

Libsndfile is actually the first program I have encountered that handles Apple Core Audio Format Files (CAF or CAFF). I haven’t even found an Apple program that creates these, at least not among the offerings bundled with Mac OS X 10.5. Now that I have created some CAFs, I see that Apple’s QuickTime Player plays them handily.

FFmpeg also doesn’t support multichannel (more-than-stereo) audio very robustly in its present incarnation. Yeah, that’s another item on the TODO list. Check out the complete specs for CAFF, however. I think if we made it a goal to support CAFF to its fullest (save perhaps for its pulse-width modulation provisions), FFmpeg’s audio handling would reign supreme at the end of the journey.

For the curious, these are the 26/120 files that FFmpeg can decode at this time:

  1. aif-alaw.aif
  2. aif-gsm610.aif
  3. aif-ima-adpcm.aif
  4. aif-pcm16.aif
  5. aif-pcm24.aif
  6. aif-pcm32.aif
  7. aif-pcms8.aif
  8. aif-ulaw.aif
  9. au-alaw.au
  10. au-pcm16.au
  11. au-pcms8.au
  12. au-ulaw.au
  13. svx-pcms8.svx
  14. voc-alaw.voc
  15. voc-pcm16.voc
  16. voc-pcmu8.voc
  17. voc-ulaw.voc
  18. wav-alaw.wav
  19. wav-gsm610.wav
  20. wav-ima-adpcm.wav
  21. wav-ms-adpcm.wav
  22. wav-pcm16.wav
  23. wav-pcm24.wav
  24. wav-pcm32.wav
  25. wav-pcmu8.wav
  26. wav-ulaw.wav

Posted in Open Source Multimedia, Python | 9 Comments »

9 Responses

  1. astrange Says:

    Pulse-width modulation is just a fancy name for 1-bit PCM at a really high sample rate – the resampler is all you need to convert it. Um, but I guess expanding 1-bit to 16-bit PCM beforehand wastes a lot of memory.

    ‘afconvert’ will make CAF files for you, and it’s a really good AAC encoder too.

  2. Diego Flameeyes Pettenò Says:

    You know, I started working on a CAFF demuxer for xine last year but stopped when I could find just a couple of samples out there. I suppose I could resume. But before I do that I should probably complete the Qt demuxer rewrite, if I do that right I can probably share code between the two.

  3. Peter Says:

    libsox (http://sox.sf.net) is another source of unsupported/documented formats.

  4. Multimedia Mike Says:

    A hearty ‘good luck’ on the QT demuxer rewrite, Flameeyes.

    Thanks for the reminder on SoX, Peter. I should repeat this same process for that library.

  5. compn Says:

    i wonder if mplayer fairs any better on these samples :)

  6. Multimedia Mike Says:

    @compn: I nominate you to find out, and to file bugs in the FFmpeg issue tracker regarding discrepancies.

  7. compn Says:

    @mike: ok , first bug reported, hopefully its valid
    https://roundup.mplayerhq.hu/roundup/ffmpeg/issue454

    also added a format so mplayer could play gsm in aif.
    http://svn.mplayerhq.hu/mplayer/trunk/etc/codecs.conf?r1=26767&r2=26779

  8. Multimedia Mike Says:

    Nice hustle. Keep it up. :-)

  9. justin Says:

    I started a CAFF demuxer for FFmpeg and submitted a patch, last year I think. libsndfile had a bug (at that time at least) in its creation of CAFF files. It had to do with the data size, IIRC. Hopefully that’s fixed now.