Image overlay over a video stream

[Nicolas Dufresne]

Le jeudi 18 octobre 2018 à 16:21 +0200, Wolfgang Grandegger a écrit :
> Hello,
> 
> I'm currently evaluating the following pipeline to receive, display
> and
> record a MJPEG video stream:
> 
>   # nice -20 \
>     gst-launch-1.0 -v \
>     udpsrc port=50004 buffer-size=180000000 do-timestamp=1 \
>       caps="application/x-rtp, media=(string)video, clock-
> rate=(int)90000, \
>       encoding-name=(string)JPEG, payload=(int)26,
> framerate=(fraction)50/1" \
>     ! rtpjitterbuffer latency=20 \
>     ! rtpjpegdepay \
>     ! vaapijpegdec \
>     ! timeoverlay \
>     ! tee name=t
>     t. ! queue ! vaapisink
>     t. ! queue ! vaapih264enc ! mp4mux ! filesink
> location=/tmp/test.mp4
> 
> The CPU usage of the various threads with and without the element
> "timeoverlay" is listed below:
> 
>     with timeoverlay ->  no     yes 
>   TID  Thread Name      CPU %  CPU %
>   ----------------------------------
>   542  gst-launch-1.0    42.0   88.8 
>   550  queue0:src         0.2    0.4
>   549  vaapiencodeh264    5.3    5.5
>   548  gmain              0.0    0.0
>   547  udpsrc0:src        8.9    8.4
>   546  rtpjitterbuffer    8.5   51.6
>   545  timer              0.0    0.0
>   544  queue1:src        14.8   17.6
>   543  queue0:src         3.9    4.9
> 
> The "timeoverlay" adds approx. 45% to the CPU load (max is 4 x 100%).
> What does take that much CPU time?

It's called software rendering (with anti-aliasing and all). There is
also a hit because you need to download/upload the pixels from/to the
GPU.

> Is there a faster way to do the overlay or is that element from the
> Pango plugin already quiet efficient?
> I want to overlay an image over the video stream, ideally done by
> the graphics hardware.

There is an active effort to enable GL rendering of CompositonOverlay
meta, but we don't have a fast method to import back GL textures into
VAAPI encoder iirc. So there is still quite some work.

> 
> Thanks for your help,
> 
> Wolfgang.
> 
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: This is a digitally signed message part
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20181018/ee80eb57/attachment.sig>