Breaking Eggs And Making Omelettes

Topics On Multimedia Technology and Reverse Engineering


Archives:

Poking At YouTube

January 4th, 2007 by Multimedia Mike

YouTube, as you probably know, is a wildly popular website that simply hosts videos that users can view by using Adobe’s Flash Player as a client. A significant part of its popularity comes from its accessibility. Users don’t have to know much about multimedia formats. They just upload whatever file they have and YouTube magically converts it to the format that they need internally in order to serve it out to other users.


YouTube Logo

I just started uploading videos to YouTube recently and decided to see just how flexible YouTube is in terms of the input formats that it accepts. In other words, I’ve been throwing lots of different files at YouTube and seeing what sticks. YouTube turns out to accept a remarkable array of obscure multimedia formats, which just happens to be my specialty. How does it handle all of these formats? I have heard rumors that it uses FFmpeg on the backend. Based on my experiments, it is more likely that it is using MEncoder, if they are using any pre-made bit of open source software. It could also be that they are using their own homegrown piece of software.

Either way, I have a strong feeling that FFmpeg contributors could feel justified in adding “Freelance YouTube Software Engineer” to their resumes/CVs. What makes me suspect this? Because I wrote FFmpeg code for a number of the atypical multimedia formats that YouTube supports and because of that, I know where the bugs are. When certain videos from the vast sample collection are converted, they exhibit the same bugs that they do under xine, FFmpeg’s ffplay, or MPlayer. However, ffplay and MPlayer behave differently on certain files and that fact offers clues about whether YouTube’s backend software might be using FFmpeg or MEncoder.

I have started a Wiki page detailing which formats that YouTube is known to support or not.

The first video I tried to upload was encoded with DOSBox’s ZMBV codec. FFmpeg has supported this codec since early last year. YouTube declined to convert it. Hmm, maybe it doesn’t use FFmpeg after all. So I went through a process with FFmpeg to convert the video to MPEG-4 with minimum quality loss and upload it, since YouTube really likes MPEG-4. Then I realized that, since I didn’t instruct FFmpeg otherwise, FFmpeg inserted its default ISO MPEG-4 video FourCC of ‘FMP4’. YouTube had no problem with that. I then tried a sample of one of my very favorite formats– Sega FILM. YouTube took it! The A/V sync was a little off, but it still worked. That’s when I went to work seeing what else YouTube could handle.

YouTube’s behaviors for Interplay MVE, Westwood VQA, 4XM, and ZyGo are all precisely consistent with MPlayer 1.0rc1 but not with FFmpeg, at least not the latest SVN copy. But I keep coming back to my baby, Sega FILM/CPK. I worked hard to sort out the details of this format and I have found a lot of quirks in different files. The different media programs that support FILM/CPK handle the quirks differently. Examine this bizarre video, for example, which involves a butterfly and a semi-automatic firearm. After the butterfly goes down, there is supposed to be more dialog while the video remains static. But the video conversion ended early. This behavior is consistent with MPlayer but not FFmpeg.

This somewhat reminds me of nmap’s method of detecting the network stack implementation for a machine on the other end of the internet– see how the stack handles a series of poorly-defined network packets.

Also, check out YouTube’s conversion of sonic3dblast_intro.avi from this location. It shows that YouTube’s software at least has Reimar Döffinger’s fix to the Duck TrueMotion v1 decoder (the great-great-great granddaddy of On2’s VP6 video codec) as outlined in this post (compare the image and note the difference in how it looked before the fix). This would indicate that YouTube’s version of the FFmpeg library — in whatever backend software they are using — is from after November, 2005 (when Reimar submitted the Duck TM1 upgrade) and before February, 2006 (when Kostya first submitted the ZMBV decoder).

One more important data point is that YouTube can also convert Indeo 4 and Indeo 5 video, and probably other codecs that are only supported, reliably, via Win32 binary codecs. FFmpeg can’t do that out of the box while MEncoder can.

I’m a little confused by the fact that a particular Id RoQ file, when converted by YouTube, has video a little ahead of the audio, while MPlayer plays the audio a bit ahead of the video. FFmpeg’s ffplay has proper sync. Though it’s essential to note that if YouTube is using code from sometime between the 11/2005-2/2006 timeframe, the current FFmpeg/MPlayer code I am experimenting with can not possibly be guaranteed to produce the same results.

I would like to take this opportunity to say that I’m not at all upset, bitter, or jealous to learn that YouTube is probably using free and open source software that I helped write in order to build one of the most popular sites on the internet which was recently purchased for a non-negligible amount of money by some search engine outfit. Further, they are perfectly within their rights to do this, and such usage adheres to the GPL/LGPL. I just found this to be a fascinating exercise and I’m amused to see how far this software has gotten. Further, I hope the new Wiki page can somehow help people create better YouTube videos (because a lot of YouTube video contributors could use the help).

Besides, now I work on the client-side software that helps to drive YouTube’s success.

See Also:

  • Google’s YouTube Uses FFmpeg: I wrote a longer post describing how I determined the software underlying YouTube’s conversion engine (includes illustrative pictures).

Posted in Reverse Engineering | 13 Comments »

13 Responses

  1. Corey Says:

    Fascinating, indeed. I made a couple short videos out of /dev/urandom: one using ffv1 and the other using snow. Youtube accepted the ffv1 file, but is taking forever on the snow one. 50 minutes have passed so far, for a 5-second file, so something must be wrong. The ffv1 file took less than 20 minutes (I didn’t see exactly when it finished).

  2. Multimedia Mike Says:

    If it’s not too much trouble, please submit your findings (pass or fail) to the new Wiki page. I am sure there are more than a few people on the internet who would be interested.

  3. Alex Says:

    Great test you have made Mike. I know of at least 3 video upload sites, which are using FFmpeg/MEncoder, one is selling his code for $80 (which is a crap :D).

    Also dont forget On2’s FlixEncoder, which leverages MEncoder for the same job.

  4. Alex Says:

    Just found this FLV example file: http://www.youtube.com/watch?v=Rsb6eejikQU&mode=user&search= (which is the zelda*.flv). It produces the same broken sound as FFmpeg currently.

  5. bcoudurier Says:

    While testing Google video, it clearly uses mencoder to transcode. I have a test which illustrates a bug in native mov demuxer.
    That file does not pass through youtube though (h264 in mov with bframes), so can’t say.

  6. Multimedia Mike Says:

    @Alex: Cool, I just uploaded that zelda sample (I’m user ‘vquant’) and I wasn’t sure what to make of that poorly converted audio.

  7. Corey Says:

    Update: The snow test failed some time last night. Results are posted on the wiki.

  8. Alexander Patrakov Says:

    Kostya, from your post, it is clear that you tested YouTube against known bugs in ffmpeg and MPlayer. However, there is one more possibility: they use a Windows machine with ffdshow plus many other installed codecs as a decoder. Is there any way to rule this out?

  9. Multimedia Mike Says:

    I’m Mike, not Kostya. Kostya lives over there. :-)

    I think we could rule out Google/YouTube’s use of a Windows-based backend because of the licensing cost they would incur for the farm of conversion machines. These people are known to use Linux wherever feasible.

    As my testing indicated, it’s most likely they are using MEncoder with binary codecs. However, we have recently learned that Google’s policy is to not confirm nor deny the usage of any open source software.

  10. dcolumbus Says:

    You know, I found this article over 2 years ago and have been looking for it ever since… and here it is!

    I wanted to pick your brain a bit. I’ve developed an application that I would like to offer uploading and converting video with. Could you possibly assist me with the most appropriate FFmpeg version? … that’s the problem with open-source… I never know which one to use!

    Thank you!

  11. Multimedia Mike Says:

    Always use the latest version SVN of FFmpeg.

  12. dcolumbus Says:

    Alright, thanks. Here’s another question… I’ve got FFMPEG installed and working wonderfully… but how can I find out whether or not the encoding has completed or not? It’s easy enough to execute the commands, but to get feedback…?

  13. Olijf Says:

    i used mencoder with their x264 codec to code a movie under Windows 7 but youtube wouldnt accept it.
    then I did the same under Ubuntu 10.04 and it worked fine, while the only difference was (I think) that the ubuntu version was installed as x64 while the windows one was an x86

    Strange behavior of one the same thing