Breaking Eggs And Making Omelettes

Topics On Multimedia Technology and Reverse Engineering


Perceptual Video Testing

June 16th, 2009 by Multimedia Mike

While I have thought a lot about the problem of testing perceptual audio decoders in FFmpeg via FATE, I have put little thought into testing perceptual video decoders. I surmise that my 1-off testing method for perceptual audio decoders is probably not tractable for video decoders. I suspect that, while rounding errors also cause off-by-1 errors in video decoders, compounded errors during multiple frames could stretch the threshold. There’s also the minor matter that it would require a much larger amount of space to store raw video data vs. the audio data required for the perceptual audio tests.

Testing video decoders isn’t such a big deal, though. The FFmpeg devs have had this problem solved via the ‘make test’ regression suite for a long time. There are a few flags to tack onto a command line which will invoke a bit exact inverse discrete cosine transform (IDCT). The trade-off is that optimized IDCT code paths are not exercised.

I’m thinking of a 2-part solution:

  1. As a first pass, enter video decoding tests that leverage the bit exact IDCT.
  2. As a second pass, craft special test cases that decode one frame of video and leverage the default, optimized IDCT. Test the result using the 1-off method.

See Also:

Posted in FATE Server | 2 Comments »

2 Responses

  1. Pengvado Says:

    There are separate functions for idct_put and idct_add. The latter is used only in inter-frames. So decoding just one frame wouldn’t cover everything.

  2. Multimedia Mike Says:

    Good point. Maybe 2 frames, verifying that the second frame is an interframe?