About that RPC-based distributed test staging idea I brainstormed yesterday, I’ll have you know that I successfully implemented the thing today. I used the fraps-v4 test spec for verification because it is known to work correctly right now, and because it only has 7 lines of stdout text. This is what the script looks like in action:

$ ./rpc-dist-test.py "FFMPEG -i 
  SAMPLES_PATH/fraps/WoW_2006-11-03_14-58-17-19-nosound-partial.avi 
  -f framecrc -" 
asking for results from 12 configurations...
testing config 0: Linux / x86_64 / gcc 4.0.4
testing config 1: Linux / x86_64 / gcc 4.1.2
testing config 2: Linux / x86_64 / gcc 4.2.4
testing config 3: Linux / x86_64 / gcc 4.3.2
testing config 4: Linux / x86_64 / gcc svn
testing config 5: Linux / PPC / gcc 4.0.4
testing config 6: Linux / PPC / gcc 4.1.2
testing config 7: Linux / PPC / gcc 4.2.4
testing config 8: Linux / PPC / gcc 4.3.2
testing config 9: Linux / PPC / gcc svn
testing config 10: Mac OS X / x86_32 / gcc 4.0.1
testing config 11: Mac OS X / x86_64 / gcc 4.0.1

1 configuration(s) failed
  configuration Mac OS X / x86_32 / gcc 4.0.1 returned status 133

There was 1 unique stdout blob collected
all successful configurations agreed on this stdout blob:
0, 0, 491520, 0x68ff12c0
0, 3000, 491520, 0x22d36f0d
0, 6000, 491520, 0xce6f877d
0, 9000, 491520, 0x85d6744c
0, 12000, 491520, 0x1aa85794
0, 15000, 491520, 0x528d1274
0, 18000, 491520, 0x357ec61c

A few notes about the foregoing:

• It doesn’t work correctly if the command is broken into multiple lines as shown; I just did that for readability.
• The Mac OS X / 32-bit configuration is not playing ball and I have yet to figure out why. The 64-bit version works great, though.
• There is a little added complexity due to my network setup with getting the Linux / 32-bit configurations to contribute results, which is why their participation isn’t shown yet.

Still, I think this is a great first run. Further, it gave me an opportunity to re-test some test specs that I had to remove from service until such time I could figure out how to make them work just right. One such spec is the ea-tgq test. This decoder relies on a bit-inexact IDCT. Unfortunately, I could not get all configurations to agree, even with the simple idct (which I don’t believe the decoder honors):

$ ./rpc-dist-test.py "FFMPEG -idct simple 
  -i SAMPLES_PATH/ea-tgq/v27.tgq 
  -an -t 0.2 -f framecrc -"
asking for results from 12 configurations...
[...]

1 configuration(s) failed
  configuration Mac OS X / x86_32 / gcc 4.0.1 returned status 133

There were 2 unique stdout blobs collected

The following configurations agree on the same stdout:
  Linux / x86_64 / gcc 4.0.4
  Linux / x86_64 / gcc 4.1.2
  Linux / x86_64 / gcc 4.3.2
  Linux / x86_64 / gcc svn
  Linux / PPC / gcc 4.0.4
  Linux / PPC / gcc 4.2.4
  Linux / PPC / gcc 4.3.2
  Linux / PPC / gcc svn
  Mac OS X / x86_64 / gcc 4.0.1
stdout:
0, 0, 34944, 0xb5d7c14a
0, 6000, 34944, 0xb5d7c14a
0, 12000, 34944, 0xb5d7c14a


The following configurations agree on the same stdout:
  Linux / x86_64 / gcc 4.2.4
  Linux / PPC / gcc 4.1.2
stdout:
0, 0, 34944, 0x5544bb6d
0, 6000, 34944, 0x5544bb6d
0, 12000, 34944, 0x5544bb6d

Interesting that those 2 particular configurations vary from the others, but agree with each other.

Not to brag, but I think the tool is already a smashing success– it has allowed me to re-instate the following test specs due to speedy verification:

• msrle-8bit
• msvideo1-16bit
• rpza