Tolerance to Errors in H.264 Decoding?

[Tim Müller]

On Mon, 2016-10-24 at 15:45 +0000, ROY Jonathan wrote:

Hi Jonathan,

> Comparing H.264 playback with GStreamer and VLC Player, the latter
> seems somewhat more tolerant to stream anomalies. In general, VLC
> Player will freeze at least partially on the last valid image until
> the next key frame is shown, whereas GStreamer will display
> (sometimes severe) gray artifacts.
> 
> Does anyone have an idea of what VLC Player is doing that GStreamer
> isn't (or vice versa) that could explain the different behavior?
> 
> Is it possible to detect decoding errors and simply drop the
> associated frames?
> 
> Could there be an option in GStreamer or its components (codec
> maybe?) to make the display more tolerant to decoding errors?

I think it all depends a bit on the context, and what the desired
behaviour is.

For what it's worth, some researchers recently came to the conclusion
that GStreamer performs better in some contexts than VLC in terms of
robustness against stream anomalies: https://gstconf.ubicast.tv/videos/
multimedia-communication-quality-assessment-testbed/ :)

It will also depend a lot on the decoder that's being used. Let's
assume you use the avdec_h264 software decoder from gst-libav (ie.
ffmpeg). We set the GST_BUFFER_FLAG_CORRUPTED on frames that have
decoding errors (ie. artefacts) *if* ffmpeg tells us this. I believe
ffmpeg only tells us about some errors but not others, so the flag may
not always be set.

You can set up a pad probe on the decoder or video sink to drop buffers
that have the flag set if that's the behaviour you prefer.

I don't know if there are decoder options to conceal errors that we
don't expose.

Cheers
 -Tim

-- 
Tim Müller, Centricular Ltd - http://www.centricular.com