video interrupted,timestamping issue?

[Katerina Voulgary]

Hello everyone,

I have these pipelines and video doesn't stream smoothly to a webrtc client:

Sender pipeline in python(goes through opencv):

         self._videoRGBString = "v4l2src device=/dev/video9 ! 
video/x-raw,format=YUY2, width=1920, height=1080,framerate=15/1 ! 
videoconvert ! queue ! video/x-raw,format=BGR ! appsink "


         self._writerRGBString = "appsrc caps=video/x-raw,format=BGR \
                                 ! queue ! videoconvert ! 
video/x-raw,format=I420,width=1920,height=1080,framerate=15/1  \
                                 ! queue ! omxh264enc profile=baseline 
control-rate=1 qp-range=35,45:35,45:35,45 peak-bitrate=1500000 
preset-level=0 \
                                 ! video/x-h264, 
stream-format=byte-stream, alignment=au, 
width=1920,height=1080,framerate=15/1\
                                 ! queue ! h264parse ! video/x-h264, 
alignment=au, stream-format=byte-stream, 
width=1920,height=1080,framerate=15/1\
                                 ! queue ! rtph264pay config-interval=-1 
! queue ! udpsink "+ self._config['janus']['url_RGB'] +"  \
                                  async=false"

cv2.VideoWriter(self._writerRGBString,0,15.0,(1920,1080))


Receiver pipeline:

     gst-launch-1.0 udpsrc port=8006 ! 
application/x-rtp,encoding-name=H264,payload=96 ! rtpjitterbuffer mode=1 
! rtph264depay ! decodebin ! videoconvert ! glimagesink

Some logs on the receiver:

0:03:54.409415392 2343207 0x5561344bb860 WARN rtpjitterbuffer 
rtpjitterbuffer.c:749:rtp_jitter_buffer_calculate_pts: backward 
timestamps at server, schedule resync
0:03:54.409648475 2343207 0x5561344bb860 INFO rtpjitterbuffer 
rtpjitterbuffer.c:780:rtp_jitter_buffer_calculate_pts: resync to time 
0:03:54.290964724, rtptime 9:57:44.938222222
0:03:55.516084124 2343207 0x5561344bb860 WARN rtpjitterbuffer 
rtpjitterbuffer.c:572:calculate_skew: delta - skew: 0:00:01.040029654 
too big, reset skew

The video stream has to go to a JanusWebrtc server. The stream 
frequently stops(we see no image or a still frame depending on the 
browser) like it is missing data. I think it is a timestamp issue that 
happens because of the latency(4" to the screen) that the processing 
introduces and the webrtc server cannot handle. The sender machine will 
never have a correct utc time if it is of interest.

Can I get an insight of what is missing to run smoothly in the client? I 
can give any other logs/info as needed.

The machine is a jetson xavier with  GStreamer 1.14.5.