Skipped frames when displaying RTP / H.264 stream

[jean-philippe]

Hi,

I'm sending live video (30FPS @ 1920x1080 YUY2) from an iMX8MM to a Linux
host that receives and displays it. Both are connected via a point-to-point
connection (not through a network).

The video displayed stutters and I'm hoping you kind people may be able to
help me understand the cause.

I ran tests with GSstreamer v1.14.5 & v1.17.0 (built from sources)

Master pipeline (using gst-rtsp-server):
v4l2src name=camera !
video/x-raw,width=1920,height=1080,format=YUY2,framerate=30/1 ! queue !
vpuenc_h264 gop-size=30 bitrate=0 ! queue ! h264parse ! identity
name=identity ! rtph264pay name=pay0

Client pipeline:
rtspsrc name=videosrc location=rtsp://172.20.1.129:8554/video3 !
application/x-rtp,media=video ! queue name=q1 ! rtph264depay !
video/x-h264,stream-format=avc ! h264parse ! identity name=identity !
avdec_h264 ! videoconvert ! queue name=q2 ! glimagesink sync=true

On both server and client, I'm using the 'identity' element to look at the
PTS of buffers and check for a jump of more than 8ms between consecutive
buffers's PTS.

On the client, libav prints the following errors every 4.2sec: 
ERROR                  libav :0:: Frame num change from 8 to 9
ERROR                  libav :0:: decode_slice_header error

When'h264parse' is in the pipeline, I detect a jump in the PTS of 33ms
(using the 'identity' element), every 4.2ms matching with the errors.

Using a fakesink and removing the decoding of the H.264, I still detect the
33ms jump in the PTS every 4.2ms, but only with 'h264parse' in the pipeline:
rtspsrc name=videosrc location=rtsp://172.20.1.129:8554/video3 !
application/x-rtp,media=video ! queue name=q1 ! rtph264depay ! h264parse !
identity name=identity ! fakesink sync=true

I detect no jump in the PTS on the server.

Pssoibly relevant the following two warnings on the client side:
FIXME        rtpjitterbuffer
gstrtpjitterbuffer.c:1535:gst_jitter_buffer_sink_parse_caps:<rtpjitterbuffer0>
Unsupported timestamp reference clock
FIXME        rtpjitterbuffer
gstrtpjitterbuffer.c:1543:gst_jitter_buffer_sink_parse_caps:<rtpjitterbuffer0>
Unsupported media clock

If I stream the H.264 received from the RTP stream to disk instead of
rendering it, the file plays back perfectly and libav does not print the
errors shown above.
gst-launch-1.0 -e rtspsrc location=rtsp://172.20.1.129:8554/video3 latency=0
! "application/x-rtp,media=video" ! queue ! rtph264depay !
video/x-h264,stream-format=byte-stream ! filesink
location=~/gstreamer_graphs/camera.h264
gst-launch-1.0 filesrc location=~/gstreamer_graphs/camera.h264 ! h264parse !
decodebin ! autovideoconvert ! autovideosink

This is my first streaming app so I lack experience, but I presume I should
be able to receive and display a near perfect video given that I'm not
streaming on the network (point-to-point) and both client and server CPU
usage is low (<20%)?

Why do I detect jumps in the PTS every 4.2ms when using h264parse on the
client?

Given the H.264 stream captured to file replays smoothly with no missing
frames and no error message from libav, what may be causing the live display
to stutter?

Thanks
JP



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/