People often ask me how they can help with this whole FATE effort. There is one very clear item I could use help on: More tests. You can help me fill in the holes in FATE’s test coverage. For those who are interested:
- Check over the FATE Test Coverage wiki page
- Find red boxes
- Think of a method involving the ‘ffmpeg’ command line tool that exercise the feature in that box, something along the lines of “ffmpeg -i sample.file -f framecrc -“, and preferably something that is expected to be bit-exact across platforms
- Tell me about the test, and where to locate the relevant sample file
My ad-hoc — but not hard and fast — rule about FATE suite samples is that I try to keep them under 2 MB. Exceptions can be made, but I often carve up samples manually, either with dd, or by remuxing, or using both methods.
If you have any concrete test spec ideas, please let me know, either through the comments or through email (on the sidebar). Remember, bit-exact tests are necessary at this point, although I hope to have a workable idea for testing non-bit-exact formats soon.
Just a sidenote, but apparently FATE still uses the old mplayherhq.hu URIs for FFmpeg SVN instead of http://svn.ffmpeg.org/ffmpeg/trunk/ Maybe that would be good to change?
Thanks for reminding me that I still need to update fate-client.py. fate-script.py (which is the main test mechanism I use, and which still hasn’t been released) has been updated. Then again, I am planning to obsolete fate-client.py and just maintain fate-script.py. Uninspired names, I know.
I’d like to help, but all codecs I worked on are either already on FATE or float-based. But I could suggest some WMV3 tests, since it is a widely used, complex and absent from “make test”.
A good try would be the files at ftp://ffmpeg.org/MPlayer/samples/V-codecs/WMV9/sb_wmv9_samples/ . Each one of them give a reproducible result for “ffmpeg -vsync 0 -strict 1 -i $(FILE) -f framecrc – “.
I have plans for testing VC-1 per the formal conformance suite. But I wonder, how much overlap is between WMV3 and VC-1? If I thoroughly test VC-1, can we consider WMV3 tested?
Yes, I gave a look at the source and it looks like that if we test VC-1 well, WMV3 should be ok.