[gstreamer-bugs] [Bug 580796] New: Improve timestamp handling in gstffmpegdec using reordered_opaque

[GStreamer (bugzilla.gnome.org)]

If you have any questions why you received this email, please see the text at
the end of this email. Replies to this email are NOT read, please see the text
at the end of this email. You can add comments to this bug at:
  http://bugzilla.gnome.org/show_bug.cgi?id=580796

  GStreamer | gst-ffmpeg | Ver: git
           Summary: Improve timestamp handling in gstffmpegdec using
                    reordered_opaque
           Product: GStreamer
           Version: git
          Platform: Other
        OS/Version: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: gst-ffmpeg
        AssignedTo: gstreamer-bugs at lists.sourceforge.net
        ReportedBy: vanista at gmail.com
         QAContact: gstreamer-bugs at lists.sourceforge.net
     GNOME version: Unspecified
   GNOME milestone: Unspecified


Please describe the problem:
Some streams containing complex video (interlaced, skipped or B-Frames) will
often come out desync with the audio, or be jittery because of timestamp
mis-ordering.

The main reason why this happens is that in the current implementation of
gstffmpegdec, there is no direct implicit link between incoming buffers and
decoded frames.
Depending on the profile, most decoders will queue some data before outputing
the first frame.
Also the chain function will reorganize incoming buffers to pad them to the
remains of previous frames, this way some timestamps can be automatically lost.
Lastly, the incoming timestamps will always be in the coded order, so if you
have a sequence of IPBBPBB frames, the P frames timestamps will be future to
the B frames, therefore they cannot be applied directly to the output.

Luckily, FFMPEG provides a neat mechanism for reordering timestamps, it is used
in ffplay to assign the proper pts on a decoded frame and works very well :

/* pass the coded picture timestamp */
ffmpegdec->context->reordered_opaque = in_timestamp;

len = avcodec_decode_video (ffmpegdec->context,
      ffmpegdec->picture, &have_data, data, size);

/* recuperate the reordered timestamp */
in_timestamp = ffmpegdec->picture->reordered_opaque;


The only problem is ensuring that in_timestamp exactly matches the coded frame.
I've tackled this by using a circular list which tracks every incoming buffer
and timestamps, then I use the decoder consumption amount (len) to move the
tail of the list and have the next timestamp ready. I also invalidate a
timestamp once it is used to prevent other frames from reusing it in cases
where a large buffer contains multiple frames.

This method is very accurate and addresses every cases of video I found to
date.


Steps to reproduce:


Actual results:


Expected results:


Does this happen every time?


Other information:


-- 
See http://bugzilla.gnome.org/page.cgi?id=email.html for more info about why you received
this email, why you can't respond via email, how to stop receiving
emails (or reduce the number you receive), and how to contact someone
if you are having problems with the system.

You can add comments to this bug at http://bugzilla.gnome.org/show_bug.cgi?id=580796.