I have been studying multimedia technology since 2000. It has been a pretty chaotic technological landscape. People who wanted to publish video on the web wondered what format to use (and occasionally sought my advice). Various fiefdoms arose around Microsoft, Apple, and Real, all hoping to claim the mantle of the standard web video format. Somewhere along the line, Macromedia “accidentally” established a standard web video format via the Flash Player (now Adobe’s).
A few years ago, Adobe (my employer, BTW) upgraded the video support in Flash Player to use the same video format that happened to sit at the top of QuickTime’s codec heap: QT-MP4/H.264/AAC. A few days ago, Microsoft announced the beta of Silverlight 3, which contains support for the same formats. After absorbing that information, it took a few days for the next thought to coalesce in my mind:
We have a standard multimedia format.
All the big players support the same multimedia stack (I think even Real Player supports the same stack). I know that’s dismaying to certain elements of the free software community who insist that Xiph’s multimedia stack is the “standard” (really! there are blessed RFCs to back it up and everything); you may not like it, but that’s the way it is:
- QT-MP4 is the standard container format, not Ogg
- H.264 is the standard video codec, not Theora (or dirac)
- AAC (and also MP3, for historical purposes) is the standard audio codec, not Vorbis
Sure, you may, in principal, have to send a dollar or 2 over to the Patent Illuminati (though highly unlikely). But it’s either that, or, you know, not have a standard video format. (And remember, the HTML5 video tag is not coming to save you.)
At least the free software enthusiast can take comfort in knowing that open source (L/GPL) efforts such as FFmpeg and x264 aim to create the very best tools that anyone can possibly use to create these formats.
Addendum: Now that I think about it, I don’t necessarily know if Silverlight 3 will transport H.264 and AAC inside of a QT-MP4 container or somehow pack it into an ASF file. That would be interesting to find out, though I have read (possibly uninformed) blog chatter excited about being able to stream the same file through Flash and Silverlight.
As Dark Shikari once said, Real has its income from crippling H.264. And indeed, their RV3/4 codecs are very crippled H.264 (especially motion compensation). Also their RMVB format does not support anything not crippled by them (maybe with the exception to AC3 and AAC).
Mike, welcome to the club, now prepare to feel hated by most of the “free software multimedia community”, at least related to GStreamer I guess. I have had some pretty nasty namecalling for saying that I don’t care about Xiph in my blog for a while.
By the way, you know for sure that QT-MP4 has so many variant that even if it’s a standard, it really shows that Dan Williams hit the nail on the head with the phrase “given 10 people and a standard, you’ll end the day with 12 or 13 differently behaving “standards-compliant†implementationsâ€. Okay he was referring to GSM/UMTS standards, but the point is acceptable nonetheless.
I had in mind to do some work on MP4 container for FFmpeg since I couldn’t find an easy way to produce files that my PlayStation3 could accept, even streamcopying Sony’s own trailers, but in the past months I had no time at all. I hope my schedule is going to be cleared up soon, and maybe I’ll be able to focus again on a work that pays for the time I would spend on that ;)
(and just to throw petrol over the fire, if I were to choose an “unencumbered†container format it would be Matroska, not Ogg!)
Well, when I started hacking on lscube related software I updated the stack to use mov/mp4 h264 and aac first for the very same reason.
Still I like better Vorbis and I’d love to have dirac a bit more established.
That said mov is a not so nasty container format (even if I’d switch to NUT or mkv anytime) but for streaming would be _quite_ nice if the above mentioned stack gets RTP and RTSP instead of crazy hacks over HTTP or selfmade protocols…
+1
And I’m pretty sure it’s well known that many people don’t like the Ogg container. Vorbis is good, Theora is pants (in terms of compression efficiency). As you note, everywhere supports h.264 and AAC. We have an excellent h.264 encoder through x264, now we just need an excellent AAC encoder.
Video on the web is still a total mess and just using QT-MP4/H.264 doesn’t necessarily avoid problems.
A recent flash plugin will play it, but H.264 in flash has only been out for a year or so, and there are still a lot of old flash installs out there. Also swf doesn’t work on many mobile devices (eg cell phones). And that’s assuming that the user hasn’t blocked the flash plugin as I normally do because it’s so damn obnoxious. I enable it only when there’s something specific I want to watch. That means i have to realize there’s a video on the page.
If you just put up a MP4 file (without swf) this will play on a mac, but windows users will need to install something (eg vlc, ffdshow). But if they have any recent video player installed then they can view divx, theora, etc, so there’s not much point in using MP4 except for Mac and iPhone users.
There’s a fairly good chance that people can view MPEG4-in-AVI (ie divx), since this has been around for seven or eight years now. This may even work on some embedded/mobile browsers, but won’t work out of the box on MacOSX.
The HTML5 video tag in Firefox is actually a big step forward, as it will allow you to take care of Firefox and Fennec users easily. With a fallback to Flash with Sorenson H.263, this combination will probably cover most viewers. So this is a practical solution, even if low quality. It’s either that or try to figure out what plugins are installed using javascript, which doesn’t always work, or hope they can view H.264, which isn’t as likely as you seem to think.
And nobody uses Realplayer or Silverlight. Seriously.
“The nice thing about standards is that you have so many to choose from.” – Andrew S. Tanenbaum
The nice thing about standards is that there are so many of them to choose from:
Other standard Formats include
Container: MPEG-TS, MPEG-PS
Container (Audio Only): LATM, ADTS
Video: H.261, MPEG-1, MPEG-2/H.262, H.263, H.263+, H.263++, MPEG-4 Visual, SVC, MVC, VC-1, VC-2 (dirac intra only)
Audio: MP1, MP2, BSAC, TwinVQ, HE-AAC, HE-AACv2, HLIN, HVXC, SLS, ALS, SSC, MPEG Surround, DST, SMR, USAC, G.711, G.711.1, G.722, G.722.1, G.722.2, G.723.1, G.726, G.727, G.728, G.729
Also just for fun:
Things that aren’t standard that seem to get a lot of love from a certain corner of the free software community (FFmpeg): ALAC, non-theora VP3, VP5/6, Indeo, Cook, Various EA video game formats, FFV1, Snow, RV30/40
Silverlight3 is using MP4 as its container
See http://alexzambelli.com/blog/2009/02/10/smooth-streaming-architecture/
They have added some of their own boxes but hopefully that can be reverse engineered and support for non MS browsers should be possible.
I’ve played with the smooth streaming files and they abuse the format:
*They use the “avc1” ftyp/major brand for VC-1 streams.
*They don’t use the dvc1 box for VC-1 streams (SMPTE RP 2025-2007)
The files to crash AtomicParsley (which has been pretty robust for me in the past). GPAC-0.4.5 (release) does not recognize them as ISO Base Media format (GPAC has been buggy in the past but 0.4.5 has been much better).
http://forum.doom9.org/showthread.php?t=145308
Kind of reminds me of what Microsoft did to Java and Kerberos.
And guess what Windows Media Player 12 in Windows 7 will play ? :>
does the flash plugin use hardware overlay (like xv) or software overlay (like x11) output ?
it seems h264 would be very very slow without overlay (which of course is now being dissolved anyways).
I’ve slowly been coming to the same realization myself, that this is what everyone across the board is supporting.
I’d still like to see Matroska get some more widespread adoption, but it’s already made far more inroads than Ogg has.
@compn: All the answers here: http://blogs.adobe.com/penguin.swf/2008/05/flash_uses_the_gpu.html
I believe Matroska will get spread because of HDRip scene and therefore DixX 7.
I have to agree with whoever “Anonymous” is up there…
WMV9 was supported by 93% of web browsers, last time I checked… All combined versions of FLV just barely edge that out, but that puts us at FLV1 video, with MP3 audio… Not the kind of “standard” I’d like to get behind.
H.264 and AAC are making impressive inroads, but in the fickle world of internet video, I wouldn’t be surprised if something “trendy” came along and took the hype and momentum away from it in short order. A substantial portion of online videos being locked-up behind a proprietary and impenetrable binary SWF obfustication makes this “standard” quite a bit less standard, and non-interoperable.
I’ve been around long enough to remember when MPEG-1 was THE standard for video on the internet, and RealVideo was only just barely getting a foothold here and there. But eventually Microsoft was able to grab the ball and run with it. Quicktime made for a halfway decent contender as well.
All you need is some other company, pushing their video products, while promising trivial or non-existent fees to bypass H.264 entirely. How about it On2? Got something up your sleeve?