Debugging video jerking problem

[Wolfgang Grandegger]

Hello,

Am 06.12.2018 um 19:58 schrieb Nicolas Dufresne:
> Le jeudi 06 décembre 2018 à 14:59 +0100, Wolfgang Grandegger a écrit :
>> Hello,
>>
>> I'm currently debugging a jerking problem. When I move from 50 to 60
>> FPS, the displayed video starts jerking. Just a little bit but it's
>> clearly visible when the camera pans. Here is my pipeline:
>>
>>          gst-launch-1.0 -v \
>>           udpsrc port=50004 buffer-size=180000000 do-timestamp=1 \
>>             caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, \
>>             encoding-name=(string)JPEG, payload=(int)26, framerate=(fraction)60/1" \
>>           ! rtpjitterbuffer latency=20 \
>>           ! rtpjpegdepay \
>>           ! vaapijpegdec \
>>           ! timeoverlay text="A Text ..."
>>           ! tee name=t
>>           t. ! queue ! vaapisink
>>           t. ! queue ! vaapih264enc ! mp4mux ! filesink location=/tmp/test.mp4
>>
>> I have connected a signal for under- and overruns to both, the display and
>> record queue. What I see when the jerking starts is that I get underruns
>> on the display (vaapisink) queue. Does that mean that the packets do not
>> arrive in time due to pre-processing time? The jerking goes away if I remove
>> the tee for recording or run the video at just 50 FPS.
>> Any ideas what could cause the jerking and how it can be debugged or even
>> avoided?
> 
> My guess would be that when going for 60 to 50 FPS you increase the
> latency. This is because H264 decoding latency is in number of frames.

Not sure if I understand you correctly. I do not change FPS on the fly.
I convert my MJPEG video from 50 to 60 FPS on the remote side with

  $ ffmpeg ... -filter:v "setpts=0.83333333*PTS" -r 60 ...

and then send it to the network. The video with 50 FPS is recorded
and displayed fine. The one with 60 FPS causes jerking on the display.
The jerking is also visible on the recorded video... hm, can I assume
that the "rtpjitterbuffer" calculated jerky timestamps then?

And am using H264 encoding to record the video, not decoding.

> So the longer the duration of a frame, the longer he latency. Ideally,
> you would through your RTP signalling make the caps changes when the
> FPS changes so that latency can be recalculated. You can also use
> pipeline latency, or processing-latency on sinks.

OK, will look into that.

Wolfgang.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20181206/5ee17632/attachment-0001.sig>