Skipped frames when displaying RTP / H.264 stream

[Nicolas Dufresne]

Le vendredi 31 janvier 2020 à 06:40 -0600, jean-philippe a écrit :
> Hi,
> 
> I'm sending live video (30FPS @ 1920x1080 YUY2) from an iMX8MM to a Linux
> host that receives and displays it. Both are connected via a point-to-point
> connection (not through a network).
> 
> The video displayed stutters and I'm hoping you kind people may be able to
> help me understand the cause.
> 
> I ran tests with GSstreamer v1.14.5 & v1.17.0 (built from sources)
> 
> Master pipeline (using gst-rtsp-server):
> v4l2src name=camera !
> video/x-raw,width=1920,height=1080,format=YUY2,framerate=30/1 ! queue !
> vpuenc_h264 gop-size=30 bitrate=0 ! queue ! h264parse ! identity
> name=identity ! rtph264pay name=pay0

Careful with this bitrate=0 setting, not using CBR with UDP streaming is looking
for troubles. You might also want to make sure you have no B-Frames.

> 
> Client pipeline:
> rtspsrc name=videosrc location=rtsp://172.20.1.129:8554/video3 !
> application/x-rtp,media=video ! queue name=q1 ! rtph264depay !
> video/x-h264,stream-format=avc ! h264parse ! identity name=identity !
> avdec_h264 ! videoconvert ! queue name=q2 ! glimagesink sync=true
> 
> On both server and client, I'm using the 'identity' element to look at the
> PTS of buffers and check for a jump of more than 8ms between consecutive
> buffers's PTS.
> 
> On the client, libav prints the following errors every 4.2sec: 
> ERROR                  libav :0:: Frame num change from 8 to 9
> ERROR                  libav :0:: decode_slice_header error

This will happen on packet lost. Even on perfect network you can loose packets
with UDP. This can happen for two reasons, a) the server stream is not perfectly
CBR (specially on keyframes). That will cause burst on the line, and may exhaust
the receiver socket buffer size. b) the receiver is spending too much time
processing and it's socket buffer get exhausted.

The main configuration is the udp-buffer-size on rtspsrc element. Though, if
that is not sufficient, you should work on your sender to smooth the
transmission. These days, I configure max-bitrate on udpsink to be twice the
bandwidth configured on my encoder (as my default goto settings).

> 
> When'h264parse' is in the pipeline, I detect a jump in the PTS of 33ms
> (using the 'identity' element), every 4.2ms matching with the errors.
> 
> Using a fakesink and removing the decoding of the H.264, I still detect the
> 33ms jump in the PTS every 4.2ms, but only with 'h264parse' in the pipeline:
> rtspsrc name=videosrc location=rtsp://172.20.1.129:8554/video3 !
> application/x-rtp,media=video ! queue name=q1 ! rtph264depay ! h264parse !
> identity name=identity ! fakesink sync=true
> 
> I detect no jump in the PTS on the server.
> 
> Pssoibly relevant the following two warnings on the client side:
> FIXME        rtpjitterbuffer
> gstrtpjitterbuffer.c:1535:gst_jitter_buffer_sink_parse_caps:<rtpjitterbuffer0>
> Unsupported timestamp reference clock
> FIXME        rtpjitterbuffer
> gstrtpjitterbuffer.c:1543:gst_jitter_buffer_sink_parse_caps:<rtpjitterbuffer0>
> Unsupported media clock
> 
> If I stream the H.264 received from the RTP stream to disk instead of
> rendering it, the file plays back perfectly and libav does not print the
> errors shown above.
> gst-launch-1.0 -e rtspsrc location=rtsp://172.20.1.129:8554/video3 latency=0
> ! "application/x-rtp,media=video" ! queue ! rtph264depay !
> video/x-h264,stream-format=byte-stream ! filesink
> location=~/gstreamer_graphs/camera.h264
> gst-launch-1.0 filesrc location=~/gstreamer_graphs/camera.h264 ! h264parse !
> decodebin ! autovideoconvert ! autovideosink
> 
> This is my first streaming app so I lack experience, but I presume I should
> be able to receive and display a near perfect video given that I'm not
> streaming on the network (point-to-point) and both client and server CPU
> usage is low (<20%)?
> 
> Why do I detect jumps in the PTS every 4.2ms when using h264parse on the
> client?
> 
> Given the H.264 stream captured to file replays smoothly with no missing
> frames and no error message from libav, what may be causing the live display
> to stutter?
> 
> Thanks
> JP
> 
> 
> 
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel