Using more threads to increase bandwidth

[Edward Hervey]

Hi,

On Mon, 2013-11-18 at 21:47 +0000, Chris Tapp wrote:
> I've got an app using GStreamer 0.10 which takes arbitrary video streams and makes images available for rendering in GLES. I'm using two pipelines, both created using parse_launch.
> 
> 1) playbin2 with video going to a fakesink;
> 2) appsrc ! ffmpefcolorspace ! videoscale ! fakesink.
> 
> The first pipleline plays the video in real time so that audio playback works.
> 
> The second is then used to convert the latest frame in the fakesink to the video format/size requested by GLES.
> 
> This is working as I want, but it gives lower frame rates than I would expect at times. I'm running on a dual core Atom with hyper-threading, so I should be able to have four threads running at a time.
> 
> The ppipeline created by playbin2 will have queues in it and so will use multiple threads. I tried adding some to the second pipeline:
> 
>   appsrc ! queue2 ! ffmpefcolorspace ! queue2 ! videoscale ! fakesink

  Use queue (and not queue2 which is for buffering and not just for
thread decoupling).

> 
> but this hasn't pushed the peak cpu use up from it's 'normal' level of
> about 265%. I was hoping that this would help as I thought keeping 6
> buffers in the pipeline would mean that the color space conversion and
> scaling would then run in different threads.

  Unless the elements have multi-buffer threading (like some
libav/ffmpeg decoders), the only (and normal) thing you have done above
is split your pipeline into 3 threads:
  appsrc ! queue
  queue ! ffmpegcolorspace ! queue
  queue ! videoscale ! fakesink

  The 265% (i.e. close to 3 times full core) you are seeing is therefore
normal.

> 
> What should I be doing to maximize CPU usage?

  With the current elements ... not much from the GStreamer side. You
could create 2 such pipelines in your application and load-balance your
input data across them (i.e. 50% to one pipeline, 50% to another).

  One thing which one *could* investigate (for 1.x) is to add support in
basetransform for parallel processing. For subclasses which are not
time-dependent (i.e. they don't depend on previous/future data for the
processing, nor depend on controlled properties) we could have parallel
calls to ::transform()/::transform_ip().
  Might be tricky to figure out automatic number of threads to start and
it would introduce latency (so you wouldn't want to use it with live
pipelines), but it would be an interesting feature.

  Finally, not that in 1.x (and I thought 0.10, but could be wrong), you
could do the scaling, and maybe even the colorspace conversion, in GLES
itself :)

   Edward

> 
> Chris Tapp
> 
> opensource at keylevel.com
> www.keylevel.com
> 
> 
> 
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20131119/a3ca5498/attachment.pgp>