Breaking Eggs And Making Omelettes

Topics On Multimedia Technology and Reverse Engineering


Memory Efficient FATE

April 29th, 2008 by Multimedia Mike

The x86_32 and x86_64 FATE build machines are actually VMware Fusion-hosted virtual machines running in Mac OS X (which, indeed, shall one day perform FATE duty as well). I decided to run with the flock and use Ubuntu for these VMs. However, even with this Mac Mini maxed out to its unofficial limit of 3 gigabytes of RAM, I couldn’t help thinking that they were wasting a lot of resources by running full GNOME desktops. Each VMware session has 1/2 GB of RAM allocated to it and either an 8 or 16 GB disk. Overkill, or so I suspected. I won’t bore you with the tales of the minimal hardware on which I first started running Linux because many of you were once there. I made it a goal to get each build machine down to using a 2 GB disk and hopefully 128 MB of RAM.

I briefly researched Linux distributions that are designed to be tiny, like Damn Small Linux. But I have grown reasonably comfortable with managing Ubuntu and would like to see what I can do to make it lean. At first, I thought I could allocate a small disk, install the desktop CD image, and then either configure minimal installation or remove lots of packages and configure for no-graphics startup after the install. It turns out that Ubuntu desktop does not have much in the way of customization options, save for timezone and keyboard layout. That’s the trade-off they present for being so mainstream and simple. But the CD also can not install if it is only given 2 GB of disk to work with. It just sort of hangs at one point.

There are 2 other install options. First is the “alternate” desktop ISO. The description here sounds promising but in practice, it tries the same desktop installation. The last option is the server installation. I am a little wary of Linux distros designated as “server” distros. I have been bitten at least once by a default server installation that was locked down way too tight, like a guard dog that won’t let me near my own junkyard.

But I give it a whirl anyway and the server version of Ubuntu turns out to be exactly what I was looking for– no graphics, no office or multimedia applications, just a minimal install that only occupies 1/3 of the 2 GB disk.

Next, I check if I can get away with allocating a meager 128 MB or RAM to each system. How to determine absolute memory usage in Linux, what with buffers and caches, etc.? ‘top’ will generally report all memory is being used anyway. Fortunately, I read about a tool called ‘free’, run in a 3-second polling mode:

$ free -o -s 3
             total       used       free     shared    buffers     cached
Mem:        253080     212020      41060          0      36836     101876
Swap:            0          0          0

Actually, I just noticed that ‘top’ also reports the same buffer and cache statistics. Anyway, I had no trouble compiling FFmpeg on x86_32 with only 128 MB of RAM. I tried the same on x86_64. When the build process got to libavcodec/motion_est.c (oh yes, you know the monster of which I speak), I watched the buffers and cached columns gradually shrink while the used column steadily rose. Eventually, the operating system decided that the process was too fat to live. The upshot is that the x86_64 VM gets 256 MB of RAM. How about that? 64-bit programs really do need twice as much memory.

There’s still the issue of disk usage. 2 GB doesn’t stretch as far as it used to. Fortunately, the FATE test suite lives on a different machine. But I need to store many different compilers and hopefully the most recent ccache output for each of the compilers. Big space management strategy here: Configure FFmpeg with ‘–disable-debug’.

Posted in FATE Server | 7 Comments »

GSoC 2008 Students

April 21st, 2008 by Multimedia Mike

Google has announced which students have earned slots for the 2008 Summer of Code. As with previous years, I don’t know whether to congratulate or console the constituents of this collection:

  • Alexander Strange: Generic frame-level multithreading support for FFmpeg
  • Bartlomiej Wolowiec: Nellymoser Encoder
  • Jai Menon: ALAC Encoder
  • Keiji Costantini: LGPL reimplementation of GPL sws_scale parts
  • Kostya: AAC-LC Encoder
  • Ramiro Polla: MLP/TrueHD encoder
  • Sascha Sommer: WMA Pro Decoder
  • Sisir Koppaka: VP3/Theora Encoder
  • Zhentan Feng: MXF Muxer

Feast your eyes on those ambitious projects. It’s going to be quite a summer.

A hearty “thanks” and “good luck to you too” go out to the registered FFmpeg mentoring crew, including Andreas Setterlind, Andreas Öman, Aurélien Jacobs, Baptiste Coudurier, Benjamin Larsson, Jean-Baptiste Kempf, Justin Ruggles, Kristian Jerpetjoen, Luca Barbato, Reimar Döffinger, and Robert Swain (2006 GSoC Alumnus). And as always, there’s the unofficial über-mentor, Michael Niedermayer, who also has the final say in whether a project’s code is ready for inclusion into the tree.

Posted in Open Source Multimedia | Comments Off on GSoC 2008 Students

BFI Boredom

April 18th, 2008 by Multimedia Mike

In the nick of time, Sisir Koppaka finished the BFI playback system and qualified for FFmpeg’s 2008 Summer of Code:

BFI Boredom

The format is used in a few multimedia-heavy games from Tsunami such as Flash Traffic: City of Angels, which is pictured above. Remember that FFmpeg does not stipulate that a supported format be at all useful, or that it come from a good game. As you can see from the sample above, not even the actors could maintain any enthusiasm through the production.

Posted in Open Source Multimedia | 2 Comments »

Portable Movie Super Player

April 15th, 2008 by Multimedia Mike

I still read the IMDb Studio Briefing everyday, though it gets a little discouraging. I sometimes wonder if there will ever be anymore interesting multimedia tech news. I should have more faith: New Movie Media Devices Predicted. Really, the story here is that IBM has developed a new, giant capacity yet very small storage method. This is one of those curious situations where they don’t mention how large capacities can possibly reach but instead express the capability in terms of how much media the thing might theoretically hold. It’s left as an exercise to the reader to decide what the average size of a ‘song’ or ‘movie’ might be and compute from there.

Remember the days when CD-ROM storage capacities were expressed in terms of how many printed documents it could hold? Later, the benchmark was number of pictures, then songs. Now it’s movies. This article cites that a device built around the memory could hold the 3500 movies or 1/2 million songs. Thus, the average movie is ~140 times larger than the average song.

The weirdest aspect of the articles floating around is that the hypothetical device would come with 3500 movies prepackaged and the consumer would purchase codes to activate individual movies.

Given recent media consumption trends, there’s little reason to doubt this strategy.

Posted in Multimedia PressWatch | 2 Comments »

Sun’s Multimedia Rumblings

April 14th, 2008 by Multimedia Mike

I’m reading fluffy press releases today about how Sun is going to work towards developing an open video codec: Sun Tackles Video Codec. The article is short on substance which is generally what earns this article a spot the Multimedia PressWatch category of this blog. Something about an Open Media Stack (OMS), perhaps correlated somehow to Open Media Commons (not to be confused with Open Media Now!).

It’s hard to find anything about this initiative that’s not a rehashed press release. But this Sun blog seems to have the most authoritative information, abstract though it may be. They present a fascinating design approach: Rather than evaluate algorithmic techniques based on their performance, evaluate them based on their legal status.

Good luck to them. Here’s a Wiki page to track it.

Posted in Multimedia PressWatch | 1 Comment »

Zero Hour

April 6th, 2008 by Multimedia Mike

Just a reminder that the (revised) Summer of Code application submission deadline is tomorrow, Monday, April 7. If you are a student and want to be considered for an FFmpeg Summer of Code project slot, you need to enter an application (one or more) into Google’s system by the end of the day on Monday. That is not, however, the deadline for qualification tasks. That comes next week.

Posted in Open Source Multimedia | Comments Off on Zero Hour

PDP-1 Multimedia

April 5th, 2008 by Multimedia Mike

I got to see a demonstration of a restored, 45 year old DEC PDP-1 computer today at the Computer History Museum in Mountain View, CA, USA. Does that sound interesting in the context of multimedia hacking? The thing could be hooked up to some kind of dot-plotting video device, and it didn’t feature any sound audio. At least, no sound hardware out of the box. Thing is, the unit was highly mod-able.

The PDP-1 hosted what is widely believed to be the first video game ever– Spacewar!. I have already written up that aspect of the experience in my Gaming Pathology blog.

Sound, however, was possible through a hardware mod. The computer had an array of LEDs and one clever hacker thought to wire 4 of these up to square wave generators, thus producing 4-channel music. This was originally programmed in the early 1960s and was demoed today. The hacker who had originally written the music engine on a PDP-1 at MIT found himself on the restoration committee many decades later. It seems MIT had donated paper tape sequences that contained musical data that played on his music engine– but the engine code had been lost. Still, he was able to reverse engineer the audio format and reimplement the engine on the original PDP-1 hardware. Sounds familiar. He even made the same point that I like to make in my multimedia technology presentations — data is more important than code.

It almost made me feel young again. Here I am, studying multimedia formats that largely only date back about 15 years to around 1993.

Posted in Reverse Engineering | 2 Comments »

Solid Snake Oggs

April 4th, 2008 by Multimedia Mike

I was studying a file called vox.dat scavenged from the GameCube version of Metal Gear Solid: The Twin Snakes, the seminal, tactical, tip-toeing game starring Solid Snake. The file contains a lot of multi-lingual subtitle strings as well as the actual English speech recited along with the subtitle presentation. What format does this commercial game use? Would you believe Ogg Vorbis? The constituent audio streams are all tagged with the string “Xiph.Org libVorbis I 20020717”, which is quite old. The current version of the Xiph’s ogg123 playback tool does not decode a stream properly. Some of the data is audible, but a lot of audio chunks are skipped. FFmpeg fares a little bit better but still scrambles some audio.

Metal Gear Solid logo

Is this a case of poor backwards compatibility? Or perhaps the creators — Silicon Knights — deliberately monkeyed with the bitstream? I found that last situation a bit implausible as I assumed developers would treat this third party codec stuff as a black box. But as an experiment, let’s go back in time, courtesy of Xiph’s source control:

svn co -r {20020717} ogg-svn
svn co -r {20020717} vorbis-svn
svn co -r {20020717} vorbis-tools-svn

I removed all of the related components on my system for good measure. With a little persistence and a lot of disabled options while building the tool set, I was finally able to get the components to build. Those old tools still have the same trouble with these Ogg Vorbis files:

$ oggdec mgs1-sample1.ogg
OggDec 1.0
Decoding "mgs1-sample1.ogg" to "mgs1-sample1.wav"
        [  1.5%]Warning: hole in data
        [  4.5%]Warning: hole in data
        [  6.5%]Warning: hole in data
        [127.5%]Warning: hole in data
        [130.5%]Warning: hole in data
        [132.5%]Warning: hole in data

Or maybe the tool is just extremely capable if it can decode more than 130% of the file.

I have placed three manually ripped samples in the archive; each is 512 KB. I would start ripping at offsets where I saw an ‘OggS’ marker that was followed soon after by the libVorbis ID string. If you care enough, have a look. And to what end? Isn’t it obvious? To create a “Learn English With Solid Snake And Friends” application.

Solid Snake and Liquid Snake

Learn handy, everyday phrases like, “I’m no rookie!” and “Don’t think! Shoot!” English speakers will be able to learn the same phrases in other languages, though they won’t have the benefit of pronunciation.

I’m still working out the details of the vox.dat file format. I have some things sorted out. Perhaps readers who know German, French, Italian, or Spanish, and who understand non-ASCII character encodings can answer whether these schemes fit any well-known encodings (I know that the 0x0A is a line break in the subtitle):

             53 70 72 69  63 68 20 6E  69 63 68 74      Sprich nicht
20 7A 75 20  6D 69 72 20  77 69 65 20  7A 75 0A 65   zu mir wie zu.e
69 6E 65 6D  20 41 6E 66  1F 0B 6E 67  65 72 21     inem Anf..nger!

             4C 61 20 66  65 72 6D 65  2C 20 6C 1F      La ferme, l.
09 2D 64 65  64 61 6E 73  20 21                     .-dedans !

             5A 69 74 74  6F 20 6C 1F  09 20 64 65      Zitto l.. de
6E 74 72 6F  21                                     ntro!

             1F 42 1F 42  41 20 71 75  1F 0F 20 65       .B.BA qu.. e
73 74 1F 08  73 20 65 73  70 65 72 61  6E 64 6F 21  st..s esperando!
21 0A 1F 42  1F 42 44 69  73 70 61 72  61 21 21     !..B.BDispara!!

Empirical analysis simply implies that a character 0x1F is followed immediately by a character not in the standard ASCII set.

Posted in Game Hacking | 8 Comments »

Novice Relations Department

April 3rd, 2008 by Multimedia Mike

The qualification tasks required of students in order to participate in FFmpeg’s Summer of Code program has finally afforded me the opportunity to create a “Small FFmpeg Tasks” page on the MultimediaWiki. Of course, the small tasks list was originally chartered to qualify students, but it need not be limited to that purpose. As I wrote on the page, it can be used for anyone who needs a good starting point for FFmpeg hacking. Further, it could be used for someone who has been away from the codebase and needs an exercise to motivate re-familiarization.

I have this — perhaps unfounded — vision that there are many lurkers out there, watching the FFmpeg project from a distance, on or off the notoriously abrasive mailing lists, hoping one day to get involved somehow, but not knowing exactly how to break in. The small tasks list is a great place to start. Maybe you don’t feel all that comfortable with what you have seen during your lurkings, perhaps because you can’t figure out what is meant with this whole “top-posting” thing for which n00bs are routinely savaged. If that’s you, you can always email me privately about getting started. I will (probably, depending on the day) be happy to tutor you on the basics for contributing some code. Email address is on the side bar.

Posted in Open Source Multimedia | 4 Comments »


April 2nd, 2008 by Multimedia Mike

FFmpeg is open source, licensed largely under the Lesser General Public License (LGPL), though certain parts, for various reasons, are General Public License (GPL), version 2. There are certain provisions about when you must provide source code if you use the program. Some programs do not follow these rules.

We have started tracking these violators in the FFmpeg Hall of Shame. Each violator is listed with a link to its case in the FFmpeg issue tracker. We’re actively trying to work with all violators to resolve these issues. The Hall of Shame seems to be reserved for particularly unresponsive violators.

Posted in Open Source Multimedia | Comments Off on Violated