{"id":1850,"date":"2009-09-24T22:29:58","date_gmt":"2009-09-25T05:29:58","guid":{"rendered":"http:\/\/multimedia.cx\/eggs\/?p=1850"},"modified":"2020-07-25T23:46:08","modified_gmt":"2020-07-26T06:46:08","slug":"multithreaded-ffmpeg-programming","status":"publish","type":"post","link":"https:\/\/multimedia.cx\/eggs\/multithreaded-ffmpeg-programming\/","title":{"rendered":"Multithreaded FFmpeg Programming"},"content":{"rendered":"

As briefly mentioned in my last Theora post<\/a>, I think FFmpeg’s<\/a> Theora decoder can exploit multiple CPUs in a few ways: 1) Perform all of the DC prediction reversals in a separate thread while the main thread is busy decoding the AC coefficients (meanwhile, I have committed an optimization where the reversal occurs immediately after DC decoding in order to exploit CPU cache); 2) create n<\/em> separate threads and assign each (num_slices \/ n)<\/em> slices to decode (where a slice is a row of the image that is 16 pixels high).<\/p>\n

So there’s the plan. Now, how to take advantage of FFmpeg’s threading API (which supports POSIX threads, Win32 threads, BeOS threads, and even OS\/2 threads)? Would it surprise you to learn that this aspect is not extensively documented? Time to reverse engineer the API.<\/p>\n

I also did some Googling regarding multithreaded FFmpeg. I mostly found forum posts complaining that FFmpeg isn’t effectively leveraging however many n<\/em> cores someone’s turbo-charged machine happens to present to the OS, as demonstrated by their CPU monitoring tool. Since I suspect this post will rise in Google’s top search hits on the topic, allow me to apologize to searchers in advance by explaining<\/strong> that multimedia processing, while certainly CPU-intensive, does not necessarily lend itself to multithreading\/multiprocessing. There are a few bits here and there in the encode or decode processes that can be parallelized but the entire operation overall tends to be rather serial.<\/p>\n

So this is the goal:
\n


\n\"Mac
\n<\/center>
\n…to see FFmpeg break through the 99.9% barrier in the CPU monitor. As an aside, it briefly struck me as ironic that people want<\/em> FFmpeg to use as much of as many available CPUs as possible but scorn<\/em> the project from my day job<\/a> for being quite capable of doing the same.<\/p>\n


\n\"Big,
\n<\/center><\/p>\n

Moving right along, let’s see what can be done about exploiting what limited multithreading opportunities that Theora affords.<\/p>\n

First off: it’s necessary to explicitly enable threading at configure-time (e.g., “–enable-pthreads” for POSIX threads on Unix flavors). Not sure why this is, but there it is.<\/p>\n

<\/p>\n

Next: FFmpeg apparently allocates a thread pool during initialization based on the -threads command line parameter. Codecs can exploit these threads by loading up an array of thread-specific context data structures and calling AVCodecContext::execute() with the context array, the number of threads to use, and a function to execute in each of those threads.<\/p>\n

Here is a working, simple example of a thread context data structure and a thread function (works circa FFmpeg SVN 20000):
\n