Breaking Eggs And Making Omelettes

Topics On Multimedia Technology and Reverse Engineering


Archives:

Supplying FFmpeg With Metadata

December 21st, 2009 by Multimedia Mike

UPDATE, 2010-06-17: This information is now maintained via the FFmpeg Metadata page on the MultimediaWiki.

While creating my automated game archiving solution, I wanted to automatically tag ALAC/M4A files with appropriate metadata while encoding using FFmpeg. Then I realized I didn’t know how to do that. I do remember that FFmpeg recently supplanted a series of specific metadata command line options (like -title and -album) with a highly generalized metadata framework that is accessed by the option ‘-metadata <key>=<value>’. The official documentation provides this lonely (and, I came to realize, useless NO-OP; more on why later) example of the option’s usage:

    `-metadata key=value'

      Set a metadata key/value pair. For example, 
      for setting the title in the output file:

      ffmpeg -i in.avi -metadata title="my title" out.flv

So this option allows you to specify absolutely any key/value metadata pair you can imagine. However, a specific muxer won’t necessarily recognize it. How can I know the specific keys that, e.g., the MOV/MP4 muxer honors? 2 methods spring to mind: Trial and error (worked out well for me at first) and reading the code (which was made a good deal easier when I already knew a few of the keys that worked).

I think it would be a good idea to document the specific metadata keys that each muxer in FFmpeg recognizes. This blog post will lead by example. This data comes from SVN revision 20910, current as of 2009-12-21.

QuickTime/MOV/MP4/M4A/et al.

I have constructed the following table of the metadata keys that libavformat/movenc.c honors. The low-level identifier column lists the atom name that the format uses to encode the data on disc, which is not interesting to most readers. For the interested but uninitiated, the notation, e.g., ‘\251nam’ indicates a 4-byte code consisting of the byte A9 in hexadecimal (or 251 in octal) followed by the ASCII characters ‘n’, ‘a’, and ‘m’.

Key iTunes field Low-level identifier
“title” Name ‘\251nam’
“author” Artist ‘\251ART’
“composer” Composer ‘\251wrt’
“album” Album ‘\251alb’
“year” Year ‘\251day’
“track” Track Number ‘trkn’
“comment” Comments ‘\251cmt’
“genre” Genre ‘\251gen’
“copyright” ?? ‘\251cpy’
“description” Description ‘desc’
“synopsis” Information dialog when selecting “Show Description” in context menu ‘ldes’
“show” Show ‘tvsh’
“episode_id” Episode ID ‘tven’
“network” ?? ‘tvnn’

Further, the MOV muxer encodes libavformat version string into the ‘\251too’ field. Here’s an example that ties together all of the presently supported metadata options, even a few video options that don’t make sense for an audio file:

  ffmpeg -i track05.wav \
    -metadata title="Track #5" \
    -metadata author="Unknown Artist" \
    -metadata composer="Composer Unknown" \
    -metadata album="Tracer Video Game Soundtrack" \
    -metadata year="1996" \
    -metadata track="5" \
    -metadata comment="This is redbook CD audio track #5 from the Windows 95 game \"Tracer\"" \
    -metadata genre="Game Soundtrack" \
    -metadata copyright="Copyright 1996 Future Endeavors, Inc." \
    -metadata description="Nifty techno background tune for a futuristic video game" \
    -metadata synopsis="Hey, is this thing on? This is where the 'synopsis' field shows up." \
    -metadata show="Tracer" \
    -metadata episode_id="102" \
    -metadata network="Some network" \
    -acodec alac \
    -y track05.m4a

The lavf version shows up in the Summary tab of the informational dialog:


iTunes files encoded with FFmpeg's libavformat

This is the Info tab of the same dialog:


iTunes Info dialog

Here is the video tab:


iTunes video metadata

And this is where the “synopsis” key shows up (selecting “Show Description” from the context menu):


iTunes description dialog

I don’t know where the “copyright” or “network” strings appear in various iTunes dialogs. But the strings are definitely encoded in the file.

ASF/WMV/WMA

FFmpeg’s ASF muxer honors the following metadata keys:

  • “title”
  • “author”
  • “copyright”
  • “comment”

AVI

FFmpeg’s AVI muxer honors the following metadata keys and maps them to these FourCCs in the file header:

  • “Title” -> “INAM”
  • “Artist” or “Author” -> “IART”
  • “Copyright” -> “ICOP”
  • “Comment” -> “ICMT”
  • “Album” -> “IPRD”
  • “Genre” -> “IGNR”
  • “Track” -> “IPRT”

Further, if the bit-exact flag is not explicitly specified (using ‘-flags bitexact’, the muxer will also encode the libavformat version string into the “ISFT” field.

Matroska

Per my reading of libavformat/matroskaenc.c, the muxer only supports 3 keys. This surprises me– I always thought MKV was supposed to be a container format to end all container formats; maybe FFmpeg’s muxer just doesn’t support many of the tags available; or perhaps the format embeds some other metadata format. Whatever the case, these are the only metadata keys that the muxer supports:

  • “title”
  • “description”
  • “language”

MP3

According to libavformat/mp3.c, the following metadata keys are supported:

  • “title”
  • “author”
  • “album”
  • “year”
  • “comment”
  • “track”
  • “genre”

MPEG Transport Streams

The libavformat/mpegtsenc.c file indicates that the muxer honors the metadata keys “title” and “language”.

NUT

FFmpeg’s NUT muxer honors the following metadata keys:

  • “title”
  • “author”
  • “copyright”

Realmedia

FFmpeg’s Realmedia muxer encodes a “CONT” chunk by concatenating certain metadata values specified on the command line. These are the recognized metadata keys:

  • “title”
  • “author”
  • “copyright”
  • “comment”

Example:

  ffmpeg -i track05.wav \
    -metadata title="This is the title" \
    -metadata author="Made by Me" \
    -metadata copyright="Copyright 2009 Me" 
    -metadata comment="An exercise in Realmedia metadata" \
    -y track05.rm

This is what the start of the file looks like in a hex editor:

0040   00 01 00 03  43 4F 4E 54  00 00 00 5F  00 00 00 11  ....CONT..._....
0050   54 68 69 73  20 69 73 20  74 68 65 20  74 69 74 6C  This is the titl
0060   65 00 0A 4D  61 64 65 20  62 79 20 4D  65 00 11 43  e..Made by Me..C
0070   6F 70 79 72  69 67 68 74  20 32 30 30  39 20 4D 65  opyright 2009 Me
0080   00 21 41 6E  20 65 78 65  72 63 69 73  65 20 69 6E  .!An exercise in
0090   20 52 65 61  6C 6D 65 64  69 61 20 6D  65 74 61 64   Realmedia metad
00A0   61 74 61 4D  44 50 52 00  00 00 9B 00  00 00 00 00  ataMDPR.........

Odds ‘n Ends

The SDP muxer honors the “title” metadata key.

The SoX native format muxer honors “comment”.

Observations and Conclusions

The craziest part I notice is that the example provided in the official documentation showed how to set metadata for a FLV file. However, the FLV muxer does not honor any metadata keys. This could be that FLV metadata is highly free-form and wholly dependent on what the client player SWF cares about. The official FLV spec describes the onMetaData tag which allows a FLV to encode metadata which will be presented to an ActionScript program via Netstream.onMetaData. Really, though, this just means that this should be the easiest format for metadata support; just iterate through each key/value pair and write it to an onMetaData tag.

I was slightly disturbed by the case inconsistency seen between a few of the muxers. A few of them (like AVI) specify “Title” while most specify “title”. However, digging in libavformat/metadata.c reveals that the default behavior is to treat the metadata keys as case-insensitive unless a certain flag is specified, which is never specified.

For completeness, it looks like we should add some other fields as well. For the MP4 muxer: ‘\141ART’ (Album Artist, vs. Artist which is ‘\251ART’), ‘\251grp’ (Grouping), ‘\251lyr’ (Lyrics), ‘tvsn’ (Season Number) and ‘tves’ (Episode Number), and perhaps any other minor options, if the mood strikes. If you see some omissions with metadata types for other formats, you may want to chime in. I also notice that Ogg and FLAC are not hooked up to the metadata API while both formats support such information.

Posted in Open Source Multimedia | 20 Comments »

20 Responses

  1. bastian Says:

    Hi,
    thanks for your highly informative article!
    This may be a dumb questions as I am not too familiar with ffmpeg (yet): can I just add metadata to an existing (mp4) video file without recoding it?
    Best regards,
    -bastian

  2. Multimedia Mike Says:

    I’m pretty sure you will need to remux the file in order to add the metadata.

  3. Luzifer42 Says:

    Hello,

    I discovered that mp3 also honors “copyright”. However this is written as ID3v2 tag, wheras the other fields you mentioned are ID3v1 tags. Maybe mp3 can do even more tags.

  4. Multimedia Mike Says:

    @Luzifer42: Thanks for the update. I should review this list (I know I added a few more QuickTime metadata tags based on this survey) and document it some place more permanent.

  5. Tomas Says:

    What about the “rating” metadata? Also knowed as “stars”.

    Windows 7 can show it on its explorer. Could I create it with ffmpeg?
    I have tried with -metadata rating=5 but doesnt work

  6. Multimedia Mike Says:

    @Tomas: That sounds about right. I don’t remember seeing any mention of rating metadata in the source code.

    Now that I think about it, it we’re talking about iTunes, I don’t know if that rating metadata is stored with the file. I think it might be stored in a separate database.

  7. ffmpeg, command line media manipulation software | NetTipsDB Says:

    [...] http://tech.groups.yahoo.com/group/mypenguin99/message/16921 http://multimedia.cx/eggs/supplying-ffmpeg-with-metadata/ [...]

  8. Bob Davies Says:

    VERY helpful post thanks :)

    Still playing with this one, but it looks like you can attach the metadata without recompressing by using the copy vcodec and acodec…

    # ffmpeg -i in.mov -vcodec copy -acodec copy -metadata title=”test title” out.mov

    Saves a lot of cycles for me as every file I tried writing metadata to did a full recompress (so saves quality too).

  9. Chris Says:

    Hey there!

    This is great, thank you so much. I have a question and a comment.

    Question: Any reason why the “year” meta tag doesn’t seem to stick? Maybe some change since the version you used for this?

    Comment: I spun my wheels a bit trying to get .mov files to retain this meta data and ended up switching to .mp4, which allows the full listing above.

    Thanks again!
    Chris

  10. Chris Says:

    Got it, if you set the ‘date’ metatag t something like ’20110112′, the year will display properly.

    Cheers,
    Chris

  11. Anentropic Says:

    Do you know any way to tell ffmpeg to write specific low-level identifiers into the output file?

    The problem I have is with mp3s and Windows Explorer – even Win 7 doesn’t recognise ID3v2 tags, so when you tag the file with ffmpeg most of the info appears blank when viewing the file in Windows Explorer.

    ffmpeg translates eg -metadata title=”The Title” into TIT2=”The Title” but Windows needs a key actually called TITLE for it to display the title.

  12. Cacasodo Says:

    Great article, Mike. Cleared up the confusion!
    ‘sodo

  13. David Says:

    Playing around with FFmpeg version 0.7.7, I figured out the Matroska muxer supports arbitrary metadata:

    $ ffmpeg -i in.mkv -vcodec copy -acodec copy -metadata something=anything -metadata ‘anything=something else’ out.mkv
    …success…
    $ ffmpeg -i out.mkv

    Metadata:
    SOMETHING : anything
    ANYTHING : something else
    ENCODER : Lavf52.110.0
    $ _

  14. Multimedia Mike Says:

    @David: Thanks, I’ll be sure to update the wiki (and scrub the other muxers to learn if more metadata functionality has been added).

  15. Tino Bellmann Says:

    Thanks for this list about FFMPEG´s metadata. Great job. That´s what I was looking for.

  16. John Says:

    Part of the problem with ffmpeg and matroska .mkv files is that there is an overlap with certain attributes handled within the format others by metadata tags and some by both.

    For example language=eng does not appear as a metadata tag on a subtitle stream, but instead sets the appropriate matroska container attribute.

    e.g. “-metadata:s:s:0 language=eng”

    does not create a metadata tag on a subtitle track as you would expect, but instead causes the corresponding stream attribute to be set:

    Stream #0:2(eng), 0, 1/1000: Subtitle: ssa (default)

    In the above the “(eng)” and the “(default)” values are attributes of the stream and not metadata tags which you would expect to appear after the stream description as:

    Metadata:
    language : eng
    default : yes

    OTOH, on an audio stream it creates the tag and manipulates the internal attributes as well, but again dependent on the tag. It is all very inconsistent and there appears to be no way to control which streams have the default attribute set and which do not. So you cannot clear the “(default)” attribute on a stream simply by using -”metadata:s:a:1 default = “, for example.

  17. vinit Says:

    Hi ,
    A very comprehensive guide for the metatags that ffmpeg can mux basd on different library . Thanks for this post.
    I have specific requirement where the video is streamed , but player does not shows the progress bar. Is it possible to add the Duration and start in meta data.

    Thanks,
    Peter Parker

  18. SGM Says:

    What is the metadata supported by MXF format.

  19. mat.alex Says:

    Tags? you want tags? look here:
    http://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/

    FFMPEG tag support seems rather paltry in the face of this. But that’s not FFMPEG’s fault as there is no video metadata standard.

    Everyone has been crying about this for years – I have just been reading on an XBMC forum how nobody can get their videos tagged.

    If metadata was just a chunk in the header dumped across de/muxers which could be picked apart with some library (eg. mediainfo) everyones’ live would be easier.

    Surely this has to be the next big thing in video development? Maybe they can all agree to to use XMP or something. I think the FLV freeform idea looks quite good too.

    As for me I am struggling to find a way to keep the file creation info (date/time scene was shot) intact across de/muxers. As far as I can see the only way to do it is learn c++ and code a custom filter/plugin for something…

    ps. this is by far the most informative guide I have read so far – thanks v. much it must have taken ages to wade through the source code.

  20. Dylan Says:

    Hi guys.

    I stumbled on the excellent info while looking for a way to keep the “creation_time” tag of the huge .avi files created by my canon pocket camera. Though I learnt the basics here, I actauly didn’t read how to transfer metadata from i to o (I don’t want to manually add metadata creation_time= every time for the hundreds of home movies made with this camera over the years…)

    I found you can use -map_metadata for that.
    Now I realy didn’t get the map_metadata syntax, but this works for me very wel for me:

    #?/bin/sh
    for f in *.AVI;
    do echo “Processing $f”
    ffmpeg -i “$f” -map_metadata 0 -vcodec libx264 -preset veryslow -crf 22 -acodec libmp3lame -aq 4 “${f%.AVI}.mkv”
    done

    The creation_time tag in the .avi shows up perfectly in the .mkv file as CREATION_TIME

    Just wanted to share, since it realy helped me.
    Give and though shall receive.

Leave a Comment

Please note: Comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.