AW: RTP video stream corruption

[Thornton, Keith]

Hi, Does the h264 source contain b-frames? It could be a time-stamping issue between DTS and PTS if b frames are involved

Von: gstreamer-devel [mailto:gstreamer-devel-bounces at lists.freedesktop.org] Im Auftrag von Joel Keller
Gesendet: Mittwoch, 10. Januar 2018 20:28
An: gstreamer-devel at lists.freedesktop.org
Betreff: RTP video stream corruption

Hello,

I have a pipeline that looks like this:

rtspsrc ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! video/x-raw,format=RGBA ! appsink

which I use to receive a 4k video stream from a camera on the LAN (or to test - a stream from localhost).  Occasionally, I see that the video stream gets 'corrupted' - moving objects get smeared & then sometimes jump back/forth in time momentarily.  It looks like perhaps some I-frame information gets dropped & the h264 decoder just does the best it can until the next I-frame.  This would be okay for something like a video-conferencing solution, but my application takes decoded frames & does object recognition & temporal tracking, and the smearing & jumping really messes with this downstream processing - it would be much better to have no frames at all than these 'corrupt' ones.

I have tried all the properties & knobs that I know of w.r.t rtspsrc ( udp-buffer-size, latency, drop-on-latency, buffer-mode, retransmissions), and have tried adding queues in the pipeline, etc..  No matter what I do, this occasionally happens when there is any other significant usage of cpu (for example a multi-job compilation).  I do know that I have enough CPU time 'on-average' to keep up with the stream rate, so it seems like with correct buffering in the right places, I shouldn't have to drop any information.

I have three questions:

1)  Is there some way to configure my gstreamer pipeline so that I don't drop this information (I suspect it is the rtsp jitterbuffer dropping, but I'm not sure), and therefore don't have any problem.

2)  If I can't eliminate this via #1, can I somehow configure different information to be dropped (essentially p-frames, but not i-frames).

3) If not #1 or #2,  is there a way I can detect downstream from the video-decoder that the frame is 'corrupt' - I think the decoder should know when something is missing from the h264 stream, right?


Thanks in advance to anyone who can help shed some light on this issue.


-Joel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20180111/d8e4e31c/attachment-0001.html>