High latency with my JPEG VAAPI pipeline

[Nicolas Dufresne]

On ven, 2020-02-07 at 12:26 +0100, Wolfgang Grandegger wrote:
> Hello,
> 
> I'm trying to understand the latency figures measured with a smartphone
> and camera for the following pipeline:
> 
>   # gst-launch-1.0 -v udpsrc port=5678 buffer-size=2000000 do-timestamp=1 \
>      caps="application/x-rtp, media=(string)video, clock-rate=(int)90000,
> encoding-name=(string)JPEG, payload=(int)26, framerate=(fraction)30/1" \
>      ! rtpjitterbuffer latency=5 \
>      ! rtpjpegdepay \
>      ! vaapijpegdec \
>      ! vaapisink
> 
> I measure around 130 ms. The latency trace with "GST_TRACERS=latency

What method to you use to measure this ?

> GST_DEBUG=GST_TRACER:7" is reporting src to sink latencies of just
> around 40 ms:

You should look at the reported latency, maybe we are reporting too much. For
this do:

  GST_TRACERS="latency\(flags=pipeline,element,reported\)"

To view the results, I suggest using gst-stats-1.0 tool form the latency logs
saved to filed.

  pipeline ... 2> trace.log
  gst-stats-1.0 trace.log

Depending on the display rate, and considering you likely have a compositer,
you'll have about 1.5 vblank latency duration added to your glass to glass
latency. vaapisink maybe use more HW blocks, so there might be some latency
added there too (I'm not too wail informed on tha taspect).

There is an extra 15ms added in the vieo sink, see processing-deadline. In your
case, I assume it should have 33ms latency + processing time, so you have about
7ms of processing time.

> 
>   0:00:08.447955233   411       0x4d3c00 TRACE             GST_TRACER :0::
> latency, src-element-id=(string)0x4f03a0, src-element=(string)udpsrc0,
> src=(string)src, sink-element-id=(string)0x51acc0, sink-element=
> (string)vaapisink0, sink=(string)sink, time=(guint64)41801197,
> ts=(guint64)8447919094;
>   0:00:08.481356539   411       0x4d3c00 TRACE             GST_TRACER :0::
> latency, src-element-id=(string)0x4f03a0, src-element=(string)udpsrc0,
> src=(string)src, sink-element-id=(string)0x51acc0, sink-element=
> (string)vaapisink0, sink=(string)sink, time=(guint64)41875079,
> ts=(guint64)8481323375;
>   0:00:08.514592194   411       0x4d3c00 TRACE             GST_TRACER :0::
> latency, src-element-id=(string)0x4f03a0, src-element=(string)udpsrc0,
> src=(string)src, sink-element-id=(string)0x51acc0, sink-element=
> (string)vaapisink0, sink=(string)sink, time=(guint64)41766241,
> ts=(guint64)8514557403;
> 
> I wonder what is responsible for the 90 ms extra latency. Of course, the
> camera and the network introduces some latency, but if I use MPV
> instead,
> 
>   # mpv --profile=low-latency --untimed jpeg.sdp
> 
> I measure just 80 ms. Still a difference of 50 ms. Any idea where that
> comes from?
> 
> TIA,
> 
> Wolfgang
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel