Sending RTP through an appsink

[Stuart Marshall]

Hi, I’m having trouble with sending RTP packets through an appsink. It seems like the packets/buffers going to the appsink are not verbatim the same as what would go to a udpsink. The buffers fed to the appsink are transported in a packetized format, maintaining packet boundaries. Instrumenation seems to indicate that the packets are the same size whether they go into a udpsink or into an appsink. However, there seems to be some other difference because the client complains about the packet contents. I saw an old thread that alluded to timestamp issues, but the thread didn’t provide details. Any suggestions on what the difference is in the packets and how it can be fixed?

Thanks,

Stuart



Details follow.

The pipeline with the appsink is as follows:

avfvideosrc ! video/x-raw,width=1280,height=720,framerate=10/1 ! videoscale ! video/x-raw,width=800,height=600 ! tee name=t ! queue ! videoconvert ! appsink name=videoappsink t. ! queue ! vtenc_h264_hw max-keyframe-interval=10 bitrate=1000 realtime=true ! video/x-h264 ! rtph264pay config-interval=1 ! rtpbin.send_rtp_sink_0 rtpbin.send_rtp_src_0 ! appsink name=videodatasink rtpbin.send_rtcp_src_0 ! appsink name=videocontrolsink sync=false async=false appsrc is-live=true do-timestamp=true stream-type=0 min-latency=0 format=time name=videocontrolsrc ! rtpbin.recv_rtcp_sink_0 

In contrast, the following pipeline with a udpsink works fine:

avfvideosrc ! video/x-raw,width=1280,height=720,framerate=10/1 ! videoscale ! video/x-raw,width=800,height=600 ! tee name=t ! queue ! videoconvert ! appsink name=videoappsink t. ! queue ! vtenc_h264_hw max-keyframe-interval=10 bitrate=1000 realtime=true ! video/x-h264 ! rtph264pay config-interval=1 ! rtpbin.send_rtp_sink_0 rtpbin.send_rtp_src_0 ! udpsink clients=SERVER_IP:SERVER_VIDEO_DATA_PORT bind-port=LOCAL_VIDEO_DATA_PORT rtpbin.send_rtcp_src_0 ! udpsink clients=SERVER_IP:SERVER_VIDEO_CONTROL_PORT bind-port=LOCAL_VIDEO_CONTROL_PORT sync=false async=false udpsrc port=LOCAL_VIDEO_CONTROL_PORT ! rtpbin.recv_rtcp_sink_0 

(The all caps parameters are replaced with real values before being run.)

On the receiving side the pipeline is as follows for both sending pipelines.

rtpbin name=rtpbin latency=0 udpsrc port=42823 ! rtpbin.recv_rtcp_sink_1 rtpbin.send_rtcp_src_1 ! udpsink port=42823 clients=SERVER_IP:62857 sync=false async=false udpsrc caps="application/x-rtp,media=(string)video,payload=(int)96,clock-rate=(int)90000,encoding-name=(string)H264" port=41713 ! rtpjitterbuffer latency=0 mode=4 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtph264depay ! h264parse config-interval=1 ! avdec_h264 ! videoconvert ! videoscale ! video/x-raw,width=800,height=480 ! xvimagesink display=:0 udpsrc port=35986 ! rtpbin.recv_rtcp_sink_0 rtpbin.send_rtcp_src_0 ! udpsink port=35986 clients=SERVER_IP:43721 sync=false async=false

(To explain a bit further, in the case of the appsink the packets/buffers are transported part of the way to the client in another transport medium, and then fed to the client via udp. In the case of the udp sending pipeline, the packets go the whole trip as regular rtp udp packets.)

On the receiving side, gstreamer is complaining as follows:

0:00:03.029896136 19917      0x20fc280 WARN         rtpjitterbuffer gstrtpjitterbuffer.c:2826:gst_rtp_jitter_buffer_chain:<rtpjitterbuffer0> warning: Received invalid RTP payload, dropping
0:00:03.029911521 19917      0x20fc280 INFO        GST_ERROR_SYSTEM gstelement.c:1879:gst_element_message_full:<rtpjitterbuffer0> posting message: Could not decode stream.
0:00:03.029925604 19917      0x20fc280 INFO        GST_ERROR_SYSTEM gstelement.c:1902:gst_element_message_full:<rtpjitterbuffer0> posted warning message: Could not decode stream.
0:00:03.029954476 19917      0x20fc280 WARN         rtpjitterbuffer gstrtpjitterbuffer.c:2826:gst_rtp_jitter_buffer_chain:<rtpjitterbuffer0> warning: Received invalid RTP payload, dropping
0:00:03.029968420 19917      0x20fc280 INFO        GST_ERROR_SYSTEM gstelement.c:1879:gst_element_message_full:<rtpjitterbuffer0> posting message: Could not decode stream.
WARNING: from element /GstPipeline:pipeline0/GstRtpJitterBuffer:rtpjitterbuffer0: Could not decode stream.
0:00:03.029981335 19917      0x20fc280 INFO        GST_ERROR_SYSTEM gstelement.c:1902:gst_element_message_full:<rtpjitterbuffer0> posted warning message: Could not decode stream.
Additional debug info:
gstrtpjitterbuffer.c(2826): gst_rtp_jitter_buffer_chain (): /GstPipeline:pipeline0/GstRtpJitterBuffer:rtpjitterbuffer0:
Received invalid RTP payload, dropping
WARNING: from element /GstPipeline:pipeline0/GstRtpJitterBuffer:rtpjitterbuffer0: Could not decode stream.
Additional debug info:
gstrtpjitterbuffer.c(2826): gst_rtp_jitter_buffer_chain (): /GstPipeline:pipeline0/GstRtpJitterBuffer:rtpjitterbuffer0:
Received invalid RTP payload, dropping
0:00:03.030505895 19917      0x20fc280 WARN         rtpjitterbuffer gstrtpjitterbuffer.c:2826:gst_rtp_jitter_buffer_chain:<rtpjitterbuffer0> warning: Received invalid RTP payload, dropping
0:00:03.030528322 19917      0x20fc280 INFO        GST_ERROR_SYSTEM gstelement.c:1879:gst_element_message_full:<rtpjitterbuffer0> posting message: Could not decode stream.
0:00:03.030541373 19917      0x20fc280 INFO        GST_ERROR_SYSTEM gstelement.c:1902:gst_element_message_full:<rtpjitterbuffer0> posted warning message: Could not decode stream.