Streaming freezes (vaapi or v4l2src)

[Víctor Jáquez]

On Thu, 16 Jul 2020 at 06:53, Timtchenko, Michael wrote:
> Hello guys,
> 
> i've encountered a very strange behavior while streaming from a v4l2src to a
> kmssink (or glimagesink) by using vaapi-components for decoding. The problem
> is, that the streaming starts for 2 seconds, then freezes for 2 seconds and
> continues again for 2 seconds and so on. This effect is only noticeable when
> using vaapi-components. If I use a pipeline with simply software decoding, the
> stream is displayed fluently.
> 
> My sending pipeline is constructed as follows:
> 
> gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video$(($1+4)) !
> video/x-raw,format=YV12,width=720,height=576  ! v4l2h264enc ! rtph264pay !
> udpsink host=${IP} port=55555

You may tune the encoder to reduce the keyframe-period. Also you might choose a
good h264 profile (main or base-constrained) for encoding.

> 
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> Redistribute latency...
> 
> The erroneous reading pipeline looks like this:
> 
> gst-launch-1.0 udpsrc port=55555
> caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,
> encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! vaapih264dec !
> vaapipostproc width=720 height=576 ! glimagesink

Perhaps a h264parser after rtph264depay would make sense. Not sure if it helps,
but in my head makes sense.

Try to avoid width and height vaapipostproc properties, they are deprecated. Use
a caps filter after it.

2 secs is a lot, but perhaps the default encoding creates large gops with big
latency. So two options: set a short keyframe-period in encoder, and in the
decoder set low-latency as true, in order to push out frames as soon as decoded
even if they are required to be kept as reference.

> 
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> Got context from element 'vaapipostproc0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayDRM\)\ vaapidisplaydrm1";
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> Redistribute latency...
> WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.
> Additional debug info:
> ../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:
> There may be a timestamping problem, or this computer is too slow.
> WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.
> Additional debug info:
> ../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:
> There may be a timestamping problem, or this computer is too slow.

This is a different thing: it says that the decoder is producing more frames
than kmssink can render (above you use glimagesink, don't know if it is
different).

You can set sync=false to kmssink to render without clock time synchronization.

> 
> If I run the following pipeline on the same system, the stream is presented
> fluently without any problems:
> 
> gst-launch-1. 0 udpsrc port=55555 caps=\"application/x-rtp,
> media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264,
> payload=(int)96\"q ! rtph264depay ! avdec_h264 ! decodebin ! videoconvert !
> kmssink
> 
> I'm not sure what exactly the problem is. Maybe the timestamping mechanism in
> vaapi has a different behavior then in the avdec_h264 as the log message says?
> I've also changed log levels but I couldn't find anything helpful.
> 
> Any help for further diagnostics is highly appreciated.
> 
> Kind regards
> Michael
> 
> 
> 
> 
> 
> Bitte beachten / Please note!
> 
> *******************************************************************************
> 
> AGCO GmbH
> Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf, Germany
> Registergericht Amtsgericht Kempten HRB 10327
> Gesch?ftsf?hrer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin, Dr. Heribert Reiter, Ekkehart Glaeser
> Vorsitzender des Aufsichtsrates: Torsten Dehner
> 
> *******************************************************************************
> 
> Diese E-Mail ist nur f?r den Empf?nger bestimmt, an den es gerichtet ist und kann vertrauliches
> bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin enthaltene Ansicht
> oder Meinungs?u?erung ist die des Autors und stellt nicht notwendigerweise die Ansicht oder
> Meinung von AGCO dar. Sind Sie nicht der Empf?nger, so haben Sie diese E-Mail irrt?mlich
> erhalten und jegliche Verwendung, Ver?ffentlichung, Weiterleitung, Abschrift oder jeglicher
> Druck dieser E-Mail ist strengstens untersagt. Weder AGCO noch der Absender ?bernehmen die
> Haftung f?r Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angeh?ngte
> Dateien (sofern vorhanden) auf Viren zu ?berpr?fen.
> 
> *******************************************************************************
> 
> This email is intended solely for the use of the individual to whom it is addressed and may contain
> confidential and/or privileged material. Any views or opinions presented are solely those of the
> author and do not necessarily represent those of AGCO. If you are not the intended recipient, be
> advised that you have received this email in error and that any use, dissemination, forwarding,
> printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any
> responsibility for viruses and it is your responsibility to scan and virus check the email and its
> attachment(s) (if any).
> 
> *******************************************************************************

> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel