H264 concealment on dropped packets

[schikin]

Hello everyone, I'm trying to build the following pipeline:

Client (webrtc) -> Janus -> RTP -> gstreamer recording

In this pipeline there are obviously delays/dropouts as packets are being
delivered to Janus over Internet. Janus does a great job of conforming to
RTX (RFC 4588) and does retransmission as expected and corretly retransmits
recovered packets towards gstreamer (confirmed with sniffer).

On gstreamer this is the pipeline I'm using:

gst-launch-1.0 -v rtpbin drop-on-latency=true latency=1000
max-dropout-time=500 buffer-mode=0 do-lost=true name=rtpbin                                         
\
     udpsrc
caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264,payload=100,profile-level-id=42e01f,packetization-mode=1"
\
            port=52016 ! rtpbin.recv_rtp_sink_0  \
     rtpbin. ! queue ! rtph264depay ! h264parse config-interval=1
disable-passthrough=true ! queue ! decodebin output-corrupt=false ! queue !
videorate ! video/x-raw,framerate=30/1 ! queue ! videoscale !
video/x-raw,width=640,height=480 ! queue ! x264enc ! matroskamux ! filesink
async=false sync=false location=video.mkv

What I'm doing here:
1. Get the video recorded with no more than 1000ms delay
2. Consider the packets missing for 500ms to be lost
3. Whenever a packet loss detected - release the remaining available buffer
further down pipeline signalling the dropout
4. Make videorate conceal/interpolate the missing frame (as far as I can
understand does that by default).
5. Output with sync=false as I don't care about the random delays - what I
want is that stream is stable (no gray/green frames)

What's the problem:
Whenever a packet drop occurs (not so often in my case, 7 packets in 10mins
for one of tests) - it completely corrupts output H264 stream (gray screen)
unit the next keyframe (enforced every 5s by Janus). I understand that H264
is not too resistent to packet drops (every frame depends on previous ones)
but what I want is:
1. Whenever the lost/corrupt frame detected it is ignored and replaced by
the previously known valid frame.
2. The following frames are treated normally as if the frame was never
missed.

Visually that should look like the picture "freezing" and then continuing
with a bit of artifacts until the next keyframe (as the following I-frame
deltas would be as if there was a missed frame). I understand that there
will be artifacts anyway - that's ok but at least it won't be a gray screen
for the next 4 seconds.





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/