Multithreaded plugin

[Sebastian Dröge]

On Fr, 2014-10-24 at 21:10 -0400, Patrick Doyle wrote:
> Hello Experts,
> I need to process live (30fps) video on a frame-by-frame basis.  The
> problem is, my algorithm takes longer than 33ms per frame to operate
> (on my fast i7).  So the natural thing to do is to split up my
> processing into 2 threads.  Since my algorithm treats each frame
> independently, the simplest thing I can think of to do (and which I
> have done in the past) is to run 3 threads - a "manager" thread and 2
> "worker" threads.  The "manager" thread would hand every other thread
> to each worker thread, and harvest the results.
> 
> So here are my questions:
> 
> Is this a common idiom in gstreamer?

Not very common at the GStreamer level itself. There are some elements
that spawn multiple threads, but none for distributing the workload over
multiple threads for parallelization.

Often this kind of stuff is done in library used by GStreamer, e.g.
libav used by gst-libav, libvpx used by the vpx plugin or x264.

> Is it common enough that there are hooks, templates, or "classes" that
> make it easy to do?

Currently there's nothing to make your specific task easier. Also worse,
you can't really use any base class for this right now other than
GstElement.

What you would need to do is to collect buffers in your chain function
and pass them off to the different threads but then directly return from
the chain function (otherwise you'll never get a second buffer).
Probably you also want to block the chain function if too many buffers
are internally queued already.
Then from your manager thread you can pass the work to your worker
threads, collect any results. And then you'll need yet another thread
(ideally) for pushing the results downstream. Additionally you will need
to make sure that all serialized events are also sent downstream from
that specific thread, and serialized queries too.
It's not going to be super complicated, but there's some manual work
you'll have to do.

For the low-level threading GLib provides platform independent
abstractions for threads, all kinds of mutexes and condition variables.
And then there's GAsyncQueue and GstDataQueue, which might be helpful
for you too.

Feel free to drop by on IRC if you have more questions, that might be
better for discussions around this than the mailing list :)

-- 
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: This is a digitally signed message part
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20141025/bed5d86d/attachment.sig>