[Bug 762509] vaapidecoder: h264: decoder stores too many pictures in the DPB before output

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=762509

--- Comment #22 from Matt Staples <staples255 at gmail.com> ---
Sorry, I haven't had time to work much on this, but I do have an update.

Correcting my previous patch so that it plays interleaved files was trivial. 
It's just a matter of adding "&& gst_vaapi_frame_store_is_complete
(priv->dpb[found_index])" after "*can_be_output = found_picture...".

I could post an updated patch that at fixes that problem. It does get both of
the attached samples to play. The reason I haven't posted an update yet is
that, while they play, frames can be played out of order if the input POC
values are out of order as they are in the attached samples.

After initial failures at trying to correct that problem, I started studying
the libav (libav) decoder to see how it handled that scenario.  Turns out it
does not properly handle out-of-order frames either.  Unlike my currently
broken solution, which actually sends them out of order, libav appears to just
drop any frames that have earlier POCs than the most recently sent frame.

A quick test with the second sample showed that vaapidecode (without my
changes) produced 73 frames, while libav produced only 66 frames.

So if libav's solution is an acceptable, I believe I could quickly update my
patch to work as well as that. If not, then maybe there's a solution patch that
uses an optional mode (e.g, "quick" vs "strict" modes, defaulting to the
current strict behavior)?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.