Author Archives: Multimedia Mike

Video Coding Concepts: YUV and RGB Colorspaces And Pixel Formats

If you have any experience in programming computer graphics, you probably know all about red/green/blue (RGB) video modes and pixel formats. Guess what? It is all useless now that you are working on video codec technology!

No, that’s not entirely true. Some video codecs operate on RGB video natively. A majority of modern codecs use some kind of YUV colorspace. We will get to that. Since many programmers are familiar with RGB pixel formats, let’s use that as a starting point.

RGB Colors

To review, computers generally display RGB pixels. These pixels have red (R), green (G), and blue (B) components to them. Here are the various combinations of R, G, and B components at their minimum (0) and maximum (255/0xFF) values:

R G B color notes:
0x00 0x00 0x00 absence of R, G, and B = full black
0x00 0x00 0xFF full blue
0x00 0xFF 0x00 full green
0x00 0xFF 0xFF
0xFF 0x00 0x00 full red
0xFF 0x00 0xFF
0xFF 0xFF 0x00
0xFF 0xFF 0xFF full R, G, and B combine to make full white

YUV Colors
If you are used to dealing with RGB colors, YUV will seem a bit unintuitive at first. What does YUV stand for? Nothing you would guess. It turns out Y stands for intensity. U stands for blue and V stands for red. U is also denoted as Cb and V is also denoted as Cr. So YUV is sometimes written as YCbCr.

Here are the various combinations of Y, U, and V components at their minimum (0) and maximum (255/0xFF) values:

Y U/
Cb		 V/
Cr		 color notes
0x00 0x00 0x00
0x00 0x00 0xFF
0x00 0xFF 0x00
0x00 0xFF 0xFF
0xFF 0x00 0x00 full green
0xFF 0x00 0xFF
0xFF 0xFF 0x00
0xFF 0xFF 0xFF
0x00 0x80 0x80 full black
0x80 0x80 0x80
0xFF 0x80 0x80 full white

So, all minimum and all maximum components do not generate intuitive (read: similar to RGB) results. In fact, all 0s in the YUV colorspace result in a dull green rather than black. That last point is useful to understand when a video is displaying a lot of green block errors– that probably means that the decoder is skipping blocks of data completely and leaving the underlying YUV data as all 0.

Further Reading:

DRM Hits Home

This is positive news when viewed from a certain perspective: MLB rips off fans who bought DRM videos. So Major League Baseball (a big deal in American sports) had a media download service that was protected by iron-fisted digital rights management. It was the type of DRM that called home for authorization to check if you still had permission to view the downloaded content. And the mothership had decided to go with a different DRM system or provider.

I wonder if this is high profile enough to set a serious example for the implications of consumer media DRM? I know it’s a wake-up call for me to remember to crack all of Apple iTunes Music Store purchases while the keys are still somewhere on my computer.

According to the discussion linked above, MLB may have relented somewhat and offered existing consumers a means of re-downloading their purchased content under the new system. So maybe this episode won’t provide much of an object lesson after all.

Wii Motion JPEG

Did you know that the Nintendo Wii can play user videos? I didn’t, but the Wii can apparently play videos through a user’s SD card. I also didn’t know what format they have to be converted into in order to be played back — Motion JPEG.


Nintendo Wii

There is a Windows program based on — what else? — FFmpeg called Wii Video 9 that handily converts anything into the accepted multimedia format. From the Sony PlayStation 3 and PSP to the Apple iPod and now the Nintendo Wii, the trend is for consumer electronics devices to be able to play user videos. But they usually support more advanced formats like MPEG-4 and H.264. I’m guessing that part of the Wii’s comparably low price is directly attributable to lower multimedia technology licensing costs.

I don’t know why I find it so humorous to read the Wii Video 9 forum posts, such as “Why are the converted files so HUGE?” I think we all need to respect that the deeper aspects of multimedia technology are not second nature to the general populace. One day, this stuff may truly be transparent and ubiquitous. Until then, Monsieur Scandragon has a point when he rants, “It doesn’t matter how bad it is at playing compressed video.. 10 seconds at BAD quality should NEVER be 25MB!!”

I have been too lazy to try the program myself. I wonder if it’s normal MJPEG data, or if it is unescaped data like that seen in the THP format common on the Nintendo GameCube. I also wonder about the audio format. Presumably, if they were too cheap to license MPEG-4 or H.264 video codecs, they’re not going to license MP3 or AAC either. I am guessing either straight PCM or their own custom ADPCM format.

Android Multimedia SDK

Google has unveiled their mighty Android platform SDK today. It apparently based uses that phone-based flavor of Java. An ergonomic Eclipse-based development environment and a software emulator are both provided for your experimentation.


Android logo

That’s nice. But let’s cut to what really matters — multimedia. The SDK specifies the Media API along with its MediaPlayer and MediaRecorder APIs. According to the AudioEncoder class, audio can be encoded to AMR-NB. The VideoEncoder class specifies H.263, H.264, and MPEG-4 SP. All pretty standard for a mobile application, I suppose.

Who handles the multimedia heavy lifting? Vitor noticed this press release and associated fluffy overview from PacketVideo

It’s Java, though, and that means obfuscated Java bytecode programs. Time for a renaissance for my Java de-obfuscator?