Latency compensation

[Wolfgang Grandegger]

Hello,

Am 12.12.2018 um 20:50 schrieb Nicolas Dufresne:
> Le mercredi 12 décembre 2018 à 20:38 +0100, Wolfgang Grandegger a
> écrit :
>> Hello,
>>
>> I'm trying to understand how the processing latency is compensated for
>> the simple pipeline:
>>
>>   $ gst-launch-1.0 -v udpsrc port=50004 buffer-size=180000000 \
>>     caps="application/x-rtp, media=(string)video, clock-rate=(int)90000,
>>     encoding-name=(string)JPEG, payload=(int)26,framerate(fraction)60/1" \
>>     ! rtpjitterbuffer latency=0
>>     ! rtpjpegdepay 
>>     ! vaapijpegdec
>>     ! vaapisink 
>>
>> GST_TRACERS="latency" reports the following latency from the src to the sink:
>>
>>   61495822
>>   61865042
>>   61219613
>>   61537702
>>
>> If I use "sync=false" on the "vaapisink", I see that the latency is just 2ms.
>> There is some latency adoption taking place? How does it work? How can I
>> change it?
> 
> We are still working on improving the latency tracer, which version of
> GStreamer are you running with ?

The latest version 1.14.4

> 
> Note that the latency reported by the latency if the effective latency.
> So it's a combination of the processing and the buffering latency. With

That is also my thinking. With "latency=20" instead of "latency=0",
the reported latency is approx. 20 ms longer.

> udpsrc, depending on which start time has been decided and how much
> latency is declared by each element, you'll always get some more
> latency then when disabling sink, which will keep the queues low, but
> with a bursty rendering, and not all image might reach the screen.

How do the elements calculate their latency?

What I find with GST_DEBUG="basesink:7":

  0:00:14.179298552 ... basesink gstbasesink.c:4509:gst_base_sink_send_event:<vaapisink0>. 
  sending event 0x7ff5940065d0 latency event: 0x7ff5940065d0, time 99:99:99.999999999, 
  seq-num 65, GstEventLatency, latency=(guint64)80000000;

I wonder where the 80ms come from?

If I add "render-delay=40" to the "vaapisink", I find 120ms in the log 
line above. I think that's the extra processing delay you mentioned.
This reduces the jerking, indeed, for the price of a longer latency.

Wolfgang.