GStreamer - pipeline design - multi-threading

Tue May 30 14:25:11 UTC 2017

Le mardi 30 mai 2017 à 11:02 +0200, Bogdan Pavkovic a écrit :
> Hi all,
> 
> I realized that an attachment cannot be seen from the archives so here is the link just in case...
> 
> https://mega.nz/#!h8wFVarA!leTOvjLg4VhBxX-wNPY3_w8jcFE-W-vspXw7FOT_SAU
> 
> Best,
> B
> 
> > On Tue, May 30, 2017 at 10:49 AM, Bogdan Pavkovic <bogprs at gmail.com> wrote:
> > Dear all,
> > 
> > Here is the diagram for your convenience.
> > I hope there is someone that can answer at least some subset of my questions.
> > 
> > Thanks again in advance.
> > 
> > Best,
> > Bogdan
> > 
> > 
> > > > On Fri, May 26, 2017 at 1:15 PM, Bogdan Pavkovic <bogprs at gmail.com> wrote:
> > > Dear all,
> > > 
> > > I am a newby to GStremer Community 
> > > but I managed to thoroughly go through ALL offered tutorials.
> > > 
> > > GOAL: I need to design a performance-critical pipeline and create a couple of new filter plugins.
> > > Any help will be appreciated.
> > > 
> > > General pipeline is divided in 3 bins: input, core, and output.
> > > For the input part we have already completed 4 options: RTP client (existing plugin), MJPEG, V2L, and FILE (existing).
> > > For the output part we use Wayland.
> > > 
> > > Core part is processing which roughly is done in 3 stages: 
> > > pre-processing, heavy processing, and some sort of Kalman filtering.
> > > Q1: My idea is to make 3 plugins that will form a larger bin. Is this the right way to go?
> > > Pre-processing plug-in need to have on-request sink pads since number of input video streams can vary between 1 and 4.
> > > Source (output) pads (on request) will be created and linked to consecutive elements on the fly as they appear.

Is there a relationship between the input streams ? Since having a
single element in the context were there is not link between each
input/output pair will just make your pipeline more complex to handle.
You will endup having N+1 thread making call into your element. You
could though use a bin just to group and manage the internal filters.

> > > Q2: Do all consecutive elements in Core bin need to have on request pads due to non-deterministic number of input videos?

If I understand correctly, you are asking if the element inside the bin
need to have request pads. If that is correct, the answer is no. You
can have a bin with request sink pads, that will use ghost pad to
interface with internal filters static sink pads. The bin can then
notify sometimes src pads by ghosting a filter static src pad.

> > > 
> > > Heavy processing plug-in needs to do heavy lifting and we intend to use multi-threading on 4 available cores.
> > > Q2: Do we need to use 2 queue elements before each of the pads of this element to separate processing of each video stream?

It depends of how you implement the parallel processing. If you can do
parallel processing on a single GstBuffer, then you don't need any
queuing. This method is usually used to achieve low latency processing
as queuing multiple buffers to feed multiple threads will introduce
latency.

> > > Q3: I understood there are tasks that can do further multi-threading. Can we use them behind a queue element?

I'm not sure I understand this question. What do you mean by tasks ?
You can of course create and manage threads in your element.

> > > Q4: If we use 4 tasks, is there a way to signal and sync all of them? i.e. Do we get a signal  when all 4 task outputs are ready?

I'm not sure this question correlate with anything in GStreamer.
GstTask is a wrapper around a thread, with proper state that let you
start/stop/pause the task and link it to a thread pool. Signalling that
data is ready has to be done by your code, could be using GCond or some
advance queue data structure.

> > > Q5: How can we configure a task to target a specific core?

This is doable in a portable way, hence not expose through the GThread
abstraction. In general, you should trust your scheduler not to move
the thread around in a way that it would degrade the performance. If
you are implementing this only for unix system, you can of course
implement your own, and use pthread API directly (see
pthread_setaffinity_np).

> > > 
> > > Finaly, Kalman plugin is not calculation intensive and can handle both streams.
> > > Q6: Can we somehow undo the queue element? Does it make sense?

What does "undoing" an element means ? If by chance you are referring
to the requires queues after demuxer elements, be aware that this is
needed because demuxers receives buffer from a single streaming thread
and does not create any new threads for the output. The queues are
needed so that each branch of the demuxer can work independently. If
you have input/output pairs in your bins, you'll have 1 streaming
thread per pair, hence each output will already work independently from
each other.

> > > 
> > > I can provide a diagram of the proposed pipeline if needed to clarify things.
> > > 
> > > Many thanks in advance.
> > > 
> > > Best regards,
> > > Bogdan
> > > 
> > 
> > 
> 
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: This is a digitally signed message part
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20170530/1a2b7584/attachment.sig>