Author Archives: Multimedia Mike

Google Chrome Automated Testing

You have probably read Google’s white paper-redefining 38-page comic describing their new browser, Google Chrome. The item that captured my attention the most (aside from myriad digs against what I do at my day job) came on page 9:


Testing Google Chrome

Admirable. They test each new build against an impressive corpus of web pages. I wondered exactly how they validated the results. The answer shows up on page 11– the rendering engine produces a schematic of what it thinks the page ought to look like, rather than rendering a bitmap and performing some kind of checksum on the result. I don’t know what the schematic looks like but I wouldn’t be surprised to see some kind of XML at work. It’s still awesome to know that the browser is so aggressively tested against such a broad corpus. It has been suggested many times that FATE should try to test all of the samples in the MPlayer samples archive.

It also makes me wonder about the possibility of FFmpeg outputting syntax parsing and validating that rather than a final bitmap (or audio waveform). That would be one way around the bit inexactness problems for certain perceptual codecs. Ironically, though, such syntax parsings would be far bulkier that just raw video frames or audio waveforms.

Obviously, Google’s most brazen paradigm with this project concerns the process vs. thread concept. I remember once being tasked with a simple cleaning job. The supervisor told me to use one particular tool for the chore. After working at it for a bit, I thought of a much better tool to use for the chore. But I struggled with whether I should use the better tool because, I thought: “It seems too obvious of a solution. There must be a reason that the supervisor did not propose it.” I used the better solution anyway and when the supervisor returned, they were impressed by my solution which they hadn’t considered.

The purpose for that vague anecdote: I wonder if people have been stuck in the threaded browser model for so long that they have gotten to thinking, “There must be a good reason that all the web browsers use a threaded model rather than a process-driven model. I’m sure the browser authors have thought a lot more about this than I have and must have good reasons not to put each page in a separate process.”

I’m eager to see what Google Chrome does for the web. And I’m eager to see what their expository style does for technology white papers.

Anti-Granularity

Pursuant to my brilliant idea of granularizing the master regression test: Why didn’t anyone tell me that the seektest rule invoked the codectest and libavtest rules, thus rendering the idea completely silly? Maybe because few people understand the regression test infrastructure, which is one reason I long to supplant it with FATE. So the granular tests are out and the master regression test spec is back in. In other FATE news, I modified individual build record pages to only display failed tests, not all tests. I just think it’s cleaner and more efficient that way. Plus, each failed test lists the last known revision where the test was known to work for the given configuration:


FATE showing failed test

I’m proud of that feature, if only because it’s the most complicated and optimized SQL query I have yet devised.

Since the build record page no longer lists all the test specs, I have added a new page which does display all FATE test specs. BTW, 119/136 of the base H.264 conformance vectors now work and are actively tested against each build.

Call For SWF and FLV Spec Feedback

At work, I have been tasked with updating the Flash Player’s SWF and FLV specs for Flash Player 10. If you have an interest the documentation of multimedia formats, give the specs a read and email me with suggestions for how the specs could be more clear (and correct). I wanted to send out this solicitation during this last round of updates but the docs still had some pesky licensing terms that scared off some folks. Those ominous licensing terms were removed as part of the Open Screen Project. Here are the current links:

I thought this call would be more appropriate for my personal, technical blog with a narrower interest rather than on my more broad corporate blog. Do me a favor, though, and don’t ask that the formal specs describe the intricate details for On2 VP6 or Nelly-Moser.

Granular Regression Tests

I have been studying the FFmpeg regressions test infrastructure ever so carefully so I can decide the best way to convert all of its functionality and test coverage into the FATE system as individual test specs. I have some ideas but I never know when I will have both the time and the motivation to implement them. In the interim, I am retiring the master ‘make test’ full regression test spec. In studying the regression architecture, it’s immediately obvious that ‘make test’ invokes these 3 sub-regression suites, which each have their own test specs now:

That should help until I break down the tests even further. Oh, and I also added a test spec for ‘ffmpeg -h’. Hey, FFmpeg got into a state once where it segfaulted on help. We don’t want to let that go uncaught again.