Breaking Eggs And Making Omelettes

Topics On Multimedia Technology and Reverse Engineering


Archives:

Revenge Of The Autobuilds

July 4th, 2007 by Multimedia Mike

Takis has been a busy FFmpeg hacker: He recently established an experimental server to automatically build the current source-controlled copy of FFmpeg and perform some rudimentary tests with the output. This is some great initiative on his part.

(Oh, and look what else Takis has been up to while no one is looking: a graph of FFmpeg code change over time.)

I have wanted to build an automated building and testing infrastructure for FFmpeg for a long time now. I got my first concept up and running late last November. I just realized that I never blogged about it although I did announce it on the ffmpeg-devel mailing list. The concept lives at http://builds.multimedia.cx/, though be advised that the script that updates it went offline in late December.

Predictably, people seemed to think the autobuild system was a good idea but that my implementation needed a lot of work. And they were right. The reason that I never blogged about it is likely that I figured I was about to deploy a better concept very soon.

It is now July and I have had months to brainstorm ideas for an improved autobuild and test infrastructure. Unfortunately, as can often happen with revision 2 of an unproven idea, I fear my concept has devolved into an exercise in architecture astronomy.


Architecture Astronomy

Read Joel Spolsky’s excellent essay, “Don’t Let Architecture Astronauts Scare You”. It’s about people who heavily theorize in the abstract but rarely accomplish anything useful. Personally, I consider it a clear indicator of architecture astronomy when a program’s fundamental paradigm revolves around the idea that, “Everything is an object (or module)!” It is my opinion that declaring everything in your architecture to be an object is the abstraction endgame (to be more specific, everything is a swappable, user-configurable module, even the central engine of the program that is supposed to coordinate everything between other modules).

I’ll explain the evolution of my autobuild idea: It started simply enough with a script that iterated through a bunch of compiler versions and ran the configure/make commands to build each. It logged stdout and stderr separately and logged general information about success/failure, SVN version, etc. into a rudimentary database table that could be simply queried with a PHP script.

I soon realized that this is wholly inadequate to the overall goals I wished to accomplish in this endeavor (building and testing on many platforms). Security is a major issue, which I blogged about before, and which I solved in the first iteration using the most paranoid policies of chroot’ing the configure/make steps and prohibiting network access during the process. Another problem is the eventuality of infinite loop bugs. Any build or test step could conceivably encounter such a condition.

This realization led me to redesign the autobuild/test system as a series of individual executable steps, all stored in a database, of which the primary script has no hardcoded knowledge. And this is where the “Everything is a module” philosophy comes into play. Unfortunately, the further I plot this out on paper, the harder it becomes because the execution module concept is too generic; it’s hard to do certain specific things. I realize I need to back off a bit on the abstraction.

Posted in FATE Server, Multimedia Goals And TODO, Programming | 2 Comments »

SNOW Bounty

June 28th, 2007 by Multimedia Mike

People have often batted around the idea of offering bounties for FFmpeg work — a preset sum of money for a discrete set of goals. Lars Täuber is making a public offer for someone to complete a specific set of goals in order to bring FFmpeg’s SNOW codec up to 1.0 viability.


Wanted: SNOW
Thanks to Glass Giant’s Wanted Poster Generator

We have never done anything quite like this so this should be a learning experience for all of us. For starters, the person aiming for the 1200 Euros ought to be qualified for the task. Ideally, the individual should — at a minimum — perform a qualification task, similar to what we enforced with the Google Summer of Code this year.

Maybe we will eventually have a proper non-profit organization established in order to administer such bounty programs as well as escrow donations. In the beginning, though, if you are interested, check in on the ffmpeg-devel list and we’ll talk.

Posted in Multimedia Goals And TODO, Open Source Multimedia | 2 Comments »

Streaming Trailer Set-Top Box

March 29th, 2007 by Multimedia Mike

So the Apple TV hit the street last week. I wasn’t aware of it until a friend just IM’d me informing me that he kept up his long-standing tradition of early adoption, putting one of these shiny new units next to his Nintendo Wii and Sony PlayStation 3.


Apple TV

Here’s the killer feature he relayed to me:

You’d like apple tv. There’s a built-in movie trailer streaming function.

To think: If Apple had released this unit 7 years ago, I might never have started down the path of multimedia hacking.

Posted in Multimedia Goals And TODO | No Comments »

Master Spreadsheet

November 5th, 2006 by Multimedia Mike

Steven Zakulec (MultimediaWiki user Dashcloud) has performed an amazing service and tested every single sample in the MPlayer repository against FFmpeg. Can you even imagine? He has provided a spreadsheet (~87 KB, OpenOffice Calc format) that details which samples work or not given a particular SVN revision of the FFmpeg repository. I think this information needs to get into a searchable & maintainable online database soon so it does not go stale.

Posted in Multimedia Goals And TODO | 3 Comments »

Weak On MPEG

February 5th, 2006 by Multimedia Mike

I’ve been working tirelessly on the MultimediaWiki for the past few days, importing all of my past technical documents. Plus, I have been unearthing all kinds of documents and notes. Fortunately, my new philosophy is to just get the information out there and organized via the Wiki rather than waiting until the information is perfectly, 100% complete and verified. Some new items:

But what about MPEG? Right now, the Wiki is lacking for information regarding various MPEG audio, video, and transport formats. This was also a shortcoming under the original tech site organization. My big problem is that I simply don’t know a lot of the technical details behind MPEG. That seems to surprise a lot of people familiar with my activities. Fact is, I like to solve problems that need to be solved. Since so many other people seem to have MPEG locked down, I don’t worry about it.

So, to all who are more familiar than I in MPEG matters: What would be a good way to organize MPEG information in the MultimediaWiki?

Posted in Multimedia Goals And TODO | No Comments »

Game Formats Division

January 3rd, 2006 by Multimedia Mike

Recently, I have been getting reports left and right from people who have successfully reverse engineered old game-oriented multimedia formats. Sometimes the reports overlap, i.e., 2 people have independently RE’d the same format. I thought I would update my list of upcoming game-related formats (here’s the last entry I wrote on the matter):

  • TGV, used in various Origin games, most notably in Privateer 2: The Darkening. I have 2 contacts who have independently RE’d this format.
  • Further, it is worth investigating how this format relates to the TGQ format found on Warcraft II: The Dark Saga for the Sega Saturn.

  • EA WVE/UV2 files, which are principly Motion JPEG files
  • Multimedia from American Laser Games titles, notably Mad Dog McCree. Again, I have 2 contacts who have independently RE’d this format. This might be the IBM PhotoMotion video codec as the manual for Mad Dog McCree credits that format.
  • BRP, used in a few Argonaut games such as Alien Odyssey. After VAG figured out Argonaut’s AVS format it was postulated that their later BRP format is also a vector quantizer. It certainly looks that way based on cursory investigations.

Posted in Multimedia Goals And TODO | 6 Comments »

Game Formats On The Horizon

January 5th, 2005 by Multimedia Mike

There are few things as discouraging in research as replicated work. You work for weeks or months or whatever in order to figure out a format, and then someone else publishes before you. To that end, I wanted to post a list of multimedia technologies that other people have RE’d. I am sitting on these and will eventually get them into the open multimedia projects:

  • proper VMD audio decoding for early Sierra games
  • Gremlin Digital Video (GDV) file demuxing and A/V decoding, used in such games as Realms of the Haunting, Normality, and Hardwar, and perhaps other titles developed by Gremlin
  • AVS files , used in Creature Shock by Argonaut software and published on various platforms; the AVS files were used in the PC and Sega Saturn versions, possibly others
  • TGV files used in various Origin games
  • EA MJPEG-like video coding format used in WVE/UVE files

Posted in Multimedia Goals And TODO, Reverse Engineering | 1 Comment »