Monthly Archives: September 2008

Scary Moments In Guru History

The Multimedia Guru, Michael Niedermayer, is widely known to possess an encyclopedic — and sometimes downright frightening — knowledge of multimedia technology, theory, and related mathematics. Check out this old mailing list thread, wherein we were trying to sort of the finer details of a reverse engineered, game-related video codec (Electronic Arts TQI, if you must know). Allow me to summarize:

  • Reverse engineer: These floats show up in the original binary decoder and it’s anyone’s guess as to what they really mean: 1.306563, 0.541196, 0.382439.
  • Michael Niedermayer: 1.3065630 = cos(pi*2/16)sqrt(2), 0.5411961 = cos(pi*6/16)sqrt(2), and “0.3824393, ROTFL, this is wrong, its certainly supposed to be: 0.3826834 (0x3EC3EF15) = cos(pi*6/16); compare: 0x3ec3cf15” (and he was right)
  • Everyone else, in unison: WTF?! You knew those numbers off the top of your head?

So that pretty much left us in slack-jawed amazement. At least, until Michael revealed his secret: ‘grep -r 5411961 MPlayer’.

The Size Issue Again

I have a confession– it has been a very long time since I created any meaningful backup of the FATE database. The thing has been operational for over 8 months now and has collected hordes of data. I already had one adventure with dramatically trimming the data size. But when I try to create a backup, it just goes on and on and on.

I finally thought to look up what MySQL facilities might be able to help me diagnose this. Here’s something: SHOW TABLE STATUS. It turns out that the table that stores the build records is nearly 600 MB large. The table that holds the test results exceeds 230 MB (at least it’s not 2.3 GB like the last time I visited the size issue). The same culprit is to blame– stdout and stderr:

mysql> SELECT
    ->   SUM(LENGTH(stdout)),
    ->   SUM(LENGTH(stderr))
    -> FROM build_record;

+---------------------+---------------------+
| SUM(LENGTH(stdout)) | SUM(LENGTH(stderr)) |
+---------------------+---------------------+
|           424930434 |           165123451 | 
+---------------------+---------------------+

The database has evolved before and it’s time for the next evolution, based on the premise that there is almost no reason whatsoever to store the stdout data. I know of one circumstance where it helps– it is useful to read the configure script output to verify configuration options (especially after I update the gcc SVN configurations). And there is definitely a good reason to keep stderr around– if the build fails, the stderr is the first stop for diagnosing the problem. Even if the build succeeds, it should be theoretically possible to compare stderr across builds to find common warnings that should be eliminated (possible future expansion).

So, my solution: Periodically retire old data. In this case, I was planning to retire stdout/stderr on build records from before September. But, well… mistakes were made and I managed to retire ALL the stdout/stderr data for all existing build records, thus further underscoring the need for responsible periodic backups. Eh, I don’t think anyone looked at that data anyway. And the table is nicely compact now:

+---------------------+---------------------+
| SUM(LENGTH(stdout)) | SUM(LENGTH(stderr)) |
+---------------------+---------------------+
|              665532 |              665532 | 
+---------------------+---------------------+

It’s actually possible to back it up now. A curious observation: MySQL’s SHOW TABLE STATUS still reports a Data_length for this table of 591471720 bytes, which concerned me. But then I spied the Data_free column which reports 588306412 bytes. I think the numbers are related.

I’m still deciding on more efficient policies for the test results table, especially since I have plans to expand to other platforms soon.

What A Nerdy Place

It’s a nerdy place to live, this northern California region commonly known as Silicon Valley. In 20 minutes of driving around on a Sunday afternoon, I saw the license plates ‘UID ZRO’ and ‘GNU FAN’. For fun, I searched and found a reference to a ‘UID ZRO’ license plate. The post refers to a truck, but it was also written 9 years ago. I guess that’s the kind of vanity plate that one holds onto. However, the blog’s picture depicts the Atlanta, GA skyline. This would imply that there are 2 different geeks in 2 different states that had the same geeky idea for a nerdy license plate.

Then there was the time I saw the Ferrari in this region with the license place ‘TREO’.

And while I’m rambling about geeked-out license plates, I would be remiss if I did not mention once seeing ‘666-FSF’ in my old town. It’s highly likely that this was just a standard issue sequential plate that had an unfortunate number. But there’s also the outside chance that the automobile owner wanted to make a statement about the Free Software Foundation.