Today, Microsoft released some framework for delivering rich internet applications called Silverlight. One of its larger selling points is the ability to stream HD-quality multimedia over the internet. Naturally, I wonder what codecs the system uses. Various press releases play up the VC-1 codec, which of course only pertains to video. I can only assume that WMA3 would be a standard audio codec, and likely WMA2.
But is that all? I started asking this question after I wondered how or if someone could possibly create a Silverlight YouTube clone. The secret to YouTube’s success is, of course, free software. The site runs FFmpeg and MEncoder (see my previous post on this matter) on top of free operating systems and web servers (and I read a forum post somewhere that alleges that Python scripts schedule the multimedia conversion jobs). Sure, you could build the same kind of site with purely Microsoft OSes and multimedia conversion software, but it would cost lots of money in software licenses. This is money that doesn’t need to be spent in, e.g., YouTube’s case and can go towards expanding the hardware infrastructure or lining executive pockets.
Maybe it will be possible to build a Silverlight-based YouTube after all. Here’s a post in a Silverlight forum that describes what codecs Silverlight is alleged to support: On the video side, there is WMV1, WMV2, WMV3, WMVA, and something called WMVC1 (not a proper FourCC): Windows Media Video Advanced Profile. For audio, the post lists WMA 1, 2, and 3, and MP3. Free software encoding support for MP3 is quite abundant. FFmpeg also has support for encoding WMA2. Out of that list of video codecs, FFmpeg can encode WMV7 and 8 (no XIntra8-type frames, of course, but the encoded streams are still compliant).
As is my custom, I have started a MultimediaWiki page tracking what is known about the multimedia formats supported in Microsoft Silverlight. It would be nice to put together a simple Silverlight platform to empirically test the capabilities. I just can’t get past thinking that Silverlight should also support raw PCM, though that’s not as sexy as MP3 for mentioning in a press release. Fairness dictates I should do the same for Adobe’s Flash Player (Silverlight’s obvious competitor), so I have started an appropriate MultimediaWiki page on that topic.
Meanwhile, there’s that Moonlight project offshoot from the Mono Project that is supposed to provide Linux support for the new hotness that is Silverlight. For some reason, the meme going around today is that MS Silverlight is officially out, and Linux has 100% perfect support (or is right around the corner). My cursory investigation leads me to believe that this is not the case. I have read stuff about the Silverlight install for Linux being a one-click affair. I have some direct experience with that type of goal so I am a tad dubious. My guess is that the one-click thing is supposed to be reliable for SuSE Linux (Microsoft partner Novell’s distribution) and hopefully adaptable, with modifications, to other distros. But based on the official status page, it’s not quite there.
But thinking about my primary obsession, multimedia codecs– how will Moonlight cope with decoding the aforementioned formats? The project status page recommends developers and early adopters download and install a particular SVN revision of FFmpeg. That can’t be relied upon to decode all the formats called for above. I have read some rumblings about how Microsoft will release binary codecs that will only be allowed to run with browser plugins. So there’s something to look forward in the binary-supporting components of the free Linux multimedia players.
I should probably mention that I help develop a program that operates in a similar space as Silverlight.
Hi Mike.
Just check Icaza’s blog[1], and there he says:
# Microsoft will make the codecs for video and audio available to users of Moonlight from their web site. The codecs will be binary codecs, and they will only be licensed for use with Moonlight on a web browser (sorry, those are the rules for the Media codecs[1]).
[1]http://tirania.org/blog/archive/2007/Sep-05.html
Licensed for use with the web browser only… noted. (wink :-) )
So I guess finally the day has arrived where we must add webbrowser-capabilities to MPlayer ;-) (I’d love to see the exact license on that and how to define a web browser – not that I think it will be better than the ones for Windows, and they should be completely unacceptable for anyone who takes them seriously – not that anyone does)…
Oh, I forgot to say… Too bad they didn’t implement the video codecs in Mono/.Net. That would be great because:
1) I could be the ultimate argument if those bytecode languages really are fast these days
2) Even with obfuscation they have some advantages for reverse-engineering (and you can always hope they forget the obfuscation one day *g*).
For my part, I’m wondering if the binary codecs even exist yet. Per my reading of the MoonLight source code, there is currently no code for interfacing to the codecs. I’ll be watching. It won’t be too hard to adapt the API since the interface code will be open source.
I’m not so sure video websites like YouTube would really be possible without ffmpeg/mencoder. Every closed-source video converter I’ve ever used does a horrible job with A/V sync, and really has to be constantly babysat. Not that mencoder is easy to wrestle into near-perfect A/V sync with various formats, but at least it’s possible.
Why should somebody interested in these codecs? WMV1, WMVA and WVC1 and WMVA1/2 are supported by lavc, it’s only X8 frame support that makes WMV2/3 decoders incomplete.
To complete WM support we need WMA3, WMA lossless and X8. All of them could be done without Silverlight.
@Kostya: Don’t forget WMA3 Voice. Then again, MS’s Silverlight support page explicitly states that the lossless and voice modes of WMA3 are not supported in Silverlight.
Multimedia Mike,
You are correct pointing out that Moonlight is not ready. What was announced on the 5th was merely that Microsoft and Novell have entered into a collaboration agreement to turn Moonlight into the Silverlight implementation for Unix systems.
Moonlight started as a proof of concept that we could build such a tool using Mono and the standard Linux libraries: Pango, FreeType, Cairo and for media we prototyped things with ffmpeg.
My team’s goal is to complete the work for Silvelright 1.0 in the next six months. The good news is that Silvelright 1.0 is an incredibly simple technology and as of today we are pretty much feature complete (modulo codecs, I will get to that in a second, and full compatibility). This is where the agreement kicks-in, Microsoft is going to give us access to their regression tests suites to ensure that Moonlight is a correct third party implementation of Silverlight.
Am not sure why people got the idea that Linux support was ready, it is certainly not part of the announcement.
As you are probably painfully aware, even if there is open source code for doing decoding for all of the formats supported by Silverlight for commercial purposes it is necessary to obtain a patent license from MPEG-LA. Since taking a license would conflict with the LGPL terms that ffmpeg is under, we are unable to redistribute an ffmpeg-based Moonlight for commercial purposes because we would be violating the LGPL requirement that we give recipients the same patent rights that we would license.
Microsoft will be providing licensed versions of those codecs for Linux users to be loaded in Moonlight.
What I like about the test suite and specifications angle is that we will be a second implementation of a given spec and will help sort out the kinks that would be single-implementation only.
As for packaging, at this point I believe we will offer two options: RPM packages and DEB package repositories (so that it integrates with all major distros installation systems, we have a fairly good track record of building binaries for lots of platforms for Mono, we will now be adding the Debian-based distros like Ubuntu to the mix).
Most distros support installing from an RPM and DEB already, but am certainly willing to work with distros to ensure that we have a very smooth installation experience.
I would love to exchange notes though if you are based in Boston ;-)
Miguel
Is a patent license from mpeg-la really required to implement Windows Media Audio and Video?
Or just the MP3 part of Silverlight?
IIRC mpeg1 patents are gone and with them most if not all the mp3 patents so…