RTP video stream corruption

[Joel Keller]

Hello,

I have a pipeline that looks like this:

rtspsrc ! rtph264depay ! h264parse ! omxh264dec ! videoconvert !
video/x-raw,format=RGBA ! appsink

which I use to receive a 4k video stream from a camera on the LAN (or to
test - a stream from localhost).  Occasionally, I see that the video stream
gets 'corrupted' - moving objects get smeared & then sometimes jump
back/forth in time momentarily.  It looks like perhaps some I-frame
information gets dropped & the h264 decoder just does the best it can until
the next I-frame.  This would be okay for something like a
video-conferencing solution, but my application takes decoded frames & does
object recognition & temporal tracking, and the smearing & jumping really
messes with this downstream processing - it would be much better to have no
frames at all than these 'corrupt' ones.

I have tried all the properties & knobs that I know of w.r.t rtspsrc (
udp-buffer-size, latency, drop-on-latency, buffer-mode, retransmissions),
and have tried adding queues in the pipeline, etc..  No matter what I do,
this occasionally happens when there is any other significant usage of cpu
(for example a multi-job compilation).  I do know that I have enough CPU
time 'on-average' to keep up with the stream rate, so it seems like with
correct buffering in the right places, I shouldn't have to drop any
information.

I have three questions:

1)  Is there some way to configure my gstreamer pipeline so that I don't
drop this information (I suspect it is the rtsp jitterbuffer dropping, but
I'm not sure), and therefore don't have any problem.

2)  If I can't eliminate this via #1, can I somehow configure different
information to be dropped (essentially p-frames, but not i-frames).

3) If not #1 or #2,  is there a way I can detect downstream from the
video-decoder that the frame is 'corrupt' - I think the decoder should know
when something is missing from the h264 stream, right?


Thanks in advance to anyone who can help shed some light on this issue.


-Joel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20180110/82da4c03/attachment.html>