Last week brought us a lot of news in the web browser space: Mozilla released Firefox 3.6 (nice fullscreen video, BTW, especially on Linux); YouTube and Vimeo grabbed headlines by announcing HTML5 video support for their video sites.
I resolved a few months ago to not bother reading so many tech news sites since they consist of 99% misinformed drivel, and I’m a happier person for that decision. But when there’s big news that can be seen as tangentially related to what I do at my day job, it gets hard to resist.
From everything I read, there was surprisingly little Flash hatred in the wake of these announcements. Really, the situation just erupted into an all-out war between the devotees of Firefox (and to a lesser extent, Opera) and supporters of Google (and to a lesser extent Apple and their Safari browser). It gets boring and repetitive in a hurry when you start reading these discussions since they all go something like this:
As you can see from the infographic, at least both sides can agree on something. I would also like to state my emphatic support for Mozilla’s principled, hardline stance against the MPEG stack for HTML5 video. Please don’t budge on your position. Stand firm on the moral high ground.
That graphic is just the beginning; there are so many problems with HTML5 video that it’s hard to know where to even begin. That’s why I need to remember to just laugh gently at its mention and move along. I only get a headache trying to understand how HTML5 video could ever have the slightest chance of mattering in the grand scheme of things.
However, a pleasant side effect of this attention is that more and more people are actually being exposed to the video tag. One nagging detail people invariably notice is that the video tag performs exceptionally poorly, likely because browsers have to deal with the exact same limitations that the Flash Player does, namely, converting decoded YUV data to RGB so that it can be plopped on a browser page. And if you try to claim that you can just download the media and use a standalone player, you continue to miss the entire point of web video.
Another aspect I have to appreciate about the debate surrounding HTML5 video is the way that it brings out the positive spirit in people. Online discussions are normally overwhelmingly negative. But advocates of the HTML5/Xiph approach truly believe this could all work out: If Apple decides to adopt the Xiph stack, and if some benevolent hardware company would churn out custom ASICs for decoding Xiph codecs, and if those ASICs were adopted in next quarter’s array of mobile computing devices and netbooks, and if Google transcodes their zillobytes of YouTube videos to the Xiph stack, and if Google throws the switch and forces the 60% of IE-using stragglers to either change browsers or go without YouTube, and if Google thereby forgoes many opportunities to monetize their videos, then absolutely! HTML5 video could totally unseat Flash video.
Okay, that’s it for me. I’m going to go back to ignoring the insular, elitist tech world at large except for the few domains in which I have some influence.
See Also:
ROFL
LOL
But I will say that on OS X at least, Flash does consistently use 2-3x more CPU than either Chrome or Safari on the exact same Youtube pages.
Well, there is another way – provide good opensource alternative to Adobe Flash, so it can be played on all platforms and it’s not only you who has to worry about performance. Then Flash will be less evil and dislike will have pure technical reasons.
Why do you consider YUV2RGB to be slow?
Well, some people do suspect that this might be exactly why Google is after On2, it might be able to offer a solution that is “good enough” for all, thus giving HTML5 a chance.
But obviously it’s all wild speculation.
If the core problem with all these technologies is converting from YUV to RGB, would it not make sense to develop a video codec that can encode RGB data? Or more likely, extend h246 to include that colourspace?
The resulting files would an extra burden for TV broadcast, but then, they’re not intended for it.
Unless there’s something I’m misunderstanding about how colourspaces work? Does YUV make compression more efficient?
Awesome post Mike.
Yeah news sites and users reading from them are definitely not something you don’t want to deal with for a while at least. But I think the end of the game for me is that people *still* think the video tag is a saviour for whatever reason. Sure sure.
And in the mean time, it got *so* important that for the past year or so, Qt4 on Gentoo forced me to install phonon to be able to install webkit, as “You wouldn’t want to not have video tag support, it ‘s part of the standard!”…
@Aninhumer
Yes, YUV makes compression more efficient. Also, if codecs used RGB, then that’s twice as many samples to process in the decoder, which just takes time there instead. (H264 already supports RGB, but people don’t use it for the aforementioned reasons.)
My question is: Why don’t browsers use hardware YUV overlays like the sane media players do?
Firefox & Chrome (don’t know for Opera, IE and Safari) are slowly but surely integrating YUV display in their browser. I encourage you to read more tech news sites!
It doesn’t happen overnight, so gives them more time and a handle of release.
> My question is: Why don’t browsers use hardware YUV overlays like the sane media players do?
Safari does on Youtube HTML5 (using 4:2:0 textures and fragment programs, not overlays), but it might fall back to something slower if there are elements on top of the video.
Of course, you just have to render the stuff on top offscreen then draw it as another texture, and Flash Player could do that too.
I like “Theora is the future!” statements, but I’m a AVC fanboy.
The Direct2D-powered Firefox builds do have completely free scaling in hardware, even if the video is perspective-transformed with SVG and there are other elements over it (including text). I think it still does YV12 to RGB32 in software though, but that’s easy to fix and not horribly expensive anyway. It can do fullscreen 1080p video effortlessly, using under one core of a modern machine.
However, it’ll take quite some time until it’s officially included in Firefox. And it currently requires Windows 7 and a DirectX10 card (in theory it should work in Vista with DX9/PS2 hardware, but that is still flaky). But when it works it works very well and is very stable. Maybe in a few years…
Also YouTube in particular is not unlikely to offer Theora. They already have 5-6 versions of each video. They have introduced new versions in the past (they keep the originals), and usually most videos were available in just a week or two. Plus they don’t really care about bandwidth and space – they use quite crap settings for h.264 already. On the other hand, I don’t think any other significant video provider could afford it.
However, back to the real, present world…
I fully agree the current HTML5 video implementations are all garbage (I suspect “nice fullscreen video, BTW, especially on Linux” is ironical – in any case on Windows it has to pause and reseek when switching, and a 2GHz Core2Duo has real trouble breaking 10fps for a 320×240 video resized to 720p – Cairo or whatever they use has the slowest resizer ever, Flash trashes it; not to mention it’s crap quality, specially when downsizing). Also Flash has the awesome hardware-assisted fullscreen resizing (a shame so few video sites get it right – maybe Adobe should do some evangelism).
Chrome doesn’t even support fullscreen, and does nearest neighborhood resizing (at least is fast, until you go blind). Its buffering logic is insanely broken (both browsers attempt to do rather strange things instead of grabbing the whole file normally like Flash does for HTTP). Not sure about Safari.
Also the “fullscreen” mode that just shows the video with default controls is mostly useless – not sure what they were thinking, I don’t think YouTube and co. would be happy without their custom controls/branding and extra features (or is the user supposed to press F11?) Also hiding to option in the right click menu due to “security concerns” is just bullshit.
And it really bothers me when people pretend Theora is competitive with h.264. Every time they release a new encoder I test it to see if they did some sort of miracle (knowing deep inside it’s impossible – the format is too braindead, it wouldn’t surprise me if MPEG1 with tweaked VLC tables, exactly specified DTC + long GOPs, and some deblocker performed much better), and… well, if have to choose between “patent freedom” and “40% less bitrate”… let’s just say “patent freedom” is not worth that much :) Definitely cheaper to buy a license than to buy space and bandwidth.
The new “activity masking” think they committed recently (see http://svn.xiph.org/experimental/derf/theora-ptalarbvorm/ ) is awfully similar to the aq thing x264 does and really did improve some of the most egregious artifacts (now at decent bitrares I think I could confuse it with Xvid fast settings and no b-frames – an outstanding achievement) but it broke ratecontrol badly :(
I don’t know what I was trying to say, other than Flash is still light years ahead on the video competition. Get a nicer codec (40% nicer, good luck with that – and no, I don’t think VP8 would cut it, certainly not with their current encoder) and an usable, performant implementation on 95% of the browsers, and then we’ll talk. I would love for that to happen (sorry Mike!) but let’s be realistic.
I’m okay if theora is an option – for content providers / websites that won’t/can’t pay any ransoms.
I’m not okay if h.264 is boycotted by Mozilla to make Theora the only option. It’s possible to sidestep the licensing issues on clients’ machines, because platform decoders can be used, or the users can download decoders as extensions – whatever.
You’re missing the crucial point: mozilla devs are lazy and can’t be arsed to implement platform-specific video support.
Mans: nothing to do with “laziness”, it’s a political stance, which they won’t change unless it kills them. They also cite technical problems such as reliance is foreign code that they can’t fix, and platform fragmentation. But it’s mostly politics.
I think they used to have working prototypes with DirectShow support, and they planned to do QuickTime and GStreamer too, until they decided they wanted to push “(royalty) free codecs”, for some definition of “(royalty) free”.
If you want to read about it you can check out http://weblogs.mozillazine.org/roc/archives/2010/01/video_freedom_a.html as well as some other entries in that blog. Also note that guy in particular is fully aware of Theora’s limitations.
Yeah… I don’t really want Firefox to expose my directshow environment to the web. Even if it means h264 support.
_None_, really not a single one of the DirectShow codecs has an update mechanism suitable for such a security-critical application.
Except maybe the Microsoft ones, but there is no Microsoft DirectShow H.264 decoder, there is only a MediaFoundation one, and that is Windows 7 only.
But even then the security of Firefox would depend on how diligently Microsoft fixes security issues in its codecs, I really can’t see how Mozilla could justify _that_, making itself completely dependent on what they consider their major competitor.
Reimar: Microsoft’s MediaFoundation filters in Windows 7 have DirectShow interfaces too, any DirectShow program can use them (however if they wanted to do this there’s no reason not to use MF directly). They are versioned and digitally signed.
Firefox already exposes security bugs in the OS, for example with the font decoder via web fonts (which has had at least one exploitable hole in the last year – that’s the reason Chrome had them disabled until recently).
The point is moot anyway since they happily allow any NPAPI plugin, even versions which have known security holes. That’s the worst security problem most browsers have today, and it’s much worse that relying in MS’ video decoders.
I don’t see how they would be (more) dependent. Firefox for Windows already requires Windows.
Firefox already relies on the system for networking, file access, GUI, etc, etc. Is Directshow really that much worse than the rest of a Windows system?
The difference is that those other dependencies are in part not avoidable or at least mean security holes for other applications like IE and almost anything else.
While I think there is a way to use Windows Media Player as a plugin I still have the impressions that (particularly DirectShow) codecs are not designed to be exposed to the web.
Or to make it shorter: IMO using DirectShow is ActiveX all over again – assuming that we all agree ActiveX is an insanity security-wise, what would make DirectShow the slightest bit better?
@Mans:
the networking stuff is nowadays fairly tight and safe.
The Gui, file etc. stuff is not exposed to unvalidated data by the browser usually.
directshow has ability to output (file writer) and then wmp has its share of exploits. so probably yeah this would be a bad idea.
unless you mean to sandbox it, but windows has shown time and time again that it is incapable of doing this.
even mplayer can output files (dumpstream) and exec commands (run)
so i think exposing either to the web is not such a good idea.
@Mans: Firefox has networking libraries.
Mike, doesn’t everybody hate Flash? I know that I viciously hate it and never install it on my Linux machines. Everywhere else Flashblock is the most important Firefox extension, it cuts Firefox idle CPU consumption by an order of magnitude once a bunch of tabs with random websites are open… :-)
Also, Flash is much too slow for my Desktop. Flash player cannot play YouTube videos on my vintage hardware in realtime, while MPlayer can do it without a hitch, even in RGB colorspace at around 20-30% CPU consumption…