Author Archives: Multimedia Mike

An Object Lesson In Database Optimization

I have a tendency to regard a database engine as a black box. I just formulate my queries and count on the engine to make them fast, somehow. I think this is similar to the faith that people tend to place in language compilers– just write the C code and the compiler will just magically optimize it. And if the code isn’t fast enough, maybe you should use a higher optimization level. Of course, a computer scientist ought to be able to analyze algorithmic running efficiency and spot opportunities for theoretical improvement, rather than relying on the compiler to insert a faster machine instruction here or there.

I started reading up on MySQL optimization strategies. There are a few things to understand about how the database works under the covers, things that are quite intuitive to anyone who has a semester of data structures and algorithms coursework. The FATE database is getting slower as it grows larger. The table growing the fastest is test_result. Each build currently generates 111 new rows, one for each active test specification.

mysql> SELECT COUNT(test_spec) 
       FROM test_result 
       WHERE build_record=6742;
+------------------+
| COUNT(test_spec) |
+------------------+
|              111 |
+------------------+
1 row in set (4.12 sec)

Continue reading →

Unholy Alliance

Ma.tt (his actual domain name), the father of the WordPress blogging system, snapped this photo at the SxSW event and it gave me a cold chill for some reason:


Blu-Ray/Java Segway

I did a little searching and realized that I had already been exposed to the idea that Blu-Ray was colluding with Java. Now it occurs to me to wonder: Has there been demand for free multimedia players to support the Java functionality necessary to play Blu-Ray discs?

How Many IDs In A Database?

Current snapshot of the FATE database:

And we’re just getting started. This might be construed as either long-term planning or silly paranoia, but I have started to wonder what it would take to overflow the id field of the test_result table. I’m not even sure how large it is. MySQL simply reports the database field as being type “int(11)”. I have read various bits of literature which do not give a definitive answer on just how many bits that is. Worst case, I am assuming 32 bits, signed, with a useful positive range around 2 billion. Suppose I ramp up to around 500 unique tests in the database (hey, with all the individual regression tests yet to be imported, as well as various official conformance suites, that’s actually a fairly conservative estimate) and add 6 more configurations to round out to 20. That means each build/test cycle will generate 500 * 20 = 10000 test results. If there are 10 cycles on an average day, that means 100,000 test results per day and 3 million per month. That would last the 31-bit range for about 715 days, or nearly 2 years.

Okay, I guess I will put off worrying about the implications for the time being. But I still need to revise the test_result table to be more efficient (i.e., quit storing the stdout field if it’s the same as was specified in the test specification).

100+ FATE Tests

FATE has been public for 2 months and I have just now reached 100 tests. It’s a nice round number. No slowing down now, though. I hope for that number to go exponential, at least up to the point that FATE carefully tests 98% of FFmpeg‘s total functionality (the last 2% will be fixing bugs that I am logging as I go).

I have also been seriously looking into turning the Mac Mini into a FATE build/test machine for Mac OS X. I’m just trying to decide if I should rush it and get the configuration onto the farm with the current infrastructure, or use it as an opportunity to revise the architecture with the various efficiency brainstorms plotted on this blog. The refactoring needs to occur before I add too many more tests. For the curious, this is what the FATE script looks like while running in a screen session; it wakes up every 15 minutes and checks for a new revision in Subversion:

[Thu Mar  6 15:08:22 2008]  no change
[Thu Mar  6 15:23:26 2008]  getting new revision = 12356
  [Thu Mar  6 15:23:34 2008] building with gcc svn 132381, built 2008-02-17
  [Thu Mar  6 15:25:04 2008] testing...
  [Thu Mar  6 15:25:41 2008] logging...
  [Thu Mar  6 15:26:17 2008] building with gcc 4.0.4
  [Thu Mar  6 15:29:09 2008] testing...
  [Thu Mar  6 15:29:42 2008] logging...
  [Thu Mar  6 15:30:07 2008] building with gcc 4.1.2
  ...

Notice the time delta between logging… and the subsequent building… That delta seems to grow more or less linearly as the number of tests increases. That’s why I’m interested in optimizing that aspect sooner than later.