Someone asked me a question in email today that I thought I would pass on to the broader group. Are there any general methods for attaching general metadata — e.g., EXIF data — to individual video frames?
![video frame metadata](/eggs/images/video-frame-metadata.png)
At first thought, this strikes me as a container-level matter. However, there could be a video codec that embedded metadata in each frame and could be stored in any generalized container format.
Personally, I have always been more aware of issues surrounding content playback vs. content creation which I admit is somewhat of a weakness in my overall multimedia knowledge. The submitter had in mind data along the lines of absolute timestamps and GPS coordinate information to be included with every frame. This may seem a tad excessive but you can never underestimate other peoples’ requirements.
Well, obviously you can use any of the image codecs like JPEG or PNG, almost all containers support MPNG and especially MJPG.
First of all, thank you for answering my mail.
And thanks again for posting it here (you even created an image to explain it!)
So, as I told you in the mail, I know it’s possible to that with the FLV container, but does ffmpeg support H.264 in FLV in ffmpeg?
What about the metadata, is it possible to store that too using ffmpeg?
Thank you again!
An XML container could do it easily. Just add GPS and absolute timestamp tags somewhere within each frame header. :)
On a more serious note, is the FLV container intended to contain h.264? I got the impression that h.264 for flash was supposed to be contained in mp4 or mov.
The FLV container is not meant to contain H.264 video or AAC audio.
Something like this was already discussed on mplayer-dev-eng (IIRC).
A good suggestion was to store GPS data in a separate stream.
Basically, you could simply store a gpx file as if it were a
text subtitle (embeded in Matroska for example). Just split each
gpx track point into its own subtitle packet and store the gpx
header inside the track extradata.
That way, you could embed a gpx file inside Matroska, and extract
it from Matroska almost lossless.
This also directly associate GPS coordinates to video frames
and audio packets (when you encounter a gpx packet, consider it
is current coordinates and it applies to every audio/video
packets up to next gpx packet).
I thought it was possible to use h.264 in FLV container. Sorry, my mistake.
Thanks for the suggestion, Aurel. I’ll try Matroska.
I found the link to mplayer-dev-eng:
http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/2008-January/055907.html
The link on mplayer-dev-eng was:
http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/2008-January/055902.html