[Bug 793058] qtdemux: add support for edit lists for fragmented files in push mode

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793058

--- Comment #5 from Alicia Boya García <aboya at igalia.com> ---
Unfortunately I'm unable to replicate the behavior I saw in #2 anymore. When I
remove the `if ()` segment.stop is set to a very low value and no frames are
pushed. But you seem not to have the same problem, as I can see frames in your
output.

I'll base my answer in theory, supposing I had a GStreamer build where this
worked like one day it did.

(In reply to Thiago Sousa Santos from comment #3)
> 
> So I guess the problem is that the pts for the edit-list is slightly off.
> Right? Just confirming this is the same you see on your MSE tests.

The PTSs shown there are perfectly fine. And there are no difference with and
without edit list in that console output because you are reading buffer time,
which in qtdemux corresponds to unedited track time (i.e. "media time" in MP4
parlance).

The difference comes into play in the GstSegment emitted before the frames, as
that declares the mapping from buffer time to streaming time.

Without edit list:

ntrrgc at homura❯ gst-launch-1.0 -v pushfilesrc location=car-20120827-86.mp4 !
qtdemux ! fakesink silent=false |egrep "segment|chain" |head -n2
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   *******
(fakesink0:sink) E (type: segment (17934), GstEventSegment,
segment=(GstSegment)"GstSegment, flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE,
rate=(double)1, applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME,
base=(guint64)0, offset=(guint64)0, start=(guint64)0,
stop=(guint64)18446744073709551615, time=(guint64)0, position=(guint64)0,
duration=(guint64)18446744073709551615;";) 0x561f0df5daa0
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   *******
(fakesink0:sink) (156 bytes, dts: 0:00:00.000000000, pts: 0:00:00.041711111,
duration: 0:00:00.041711111, offset: -1, offset_end: -1, flags: 00004040
discont tag-memory , meta: none) 0x7f58e800a400

With edit list:

ntrrgc at homura❯ gst-launch-1.0 -v pushfilesrc location=car-20120827-86.mp4 !
qtdemux ! fakesink silent=false |egrep "segment|chain" |head -n3
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   *******
(fakesink0:sink) E (type: segment (17934), GstEventSegment,
segment=(GstSegment)"GstSegment, flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE,
rate=(double)1, applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME,
base=(guint64)0, offset=(guint64)0, start=(guint64)0,
stop=(guint64)18446744073709551615, time=(guint64)0, position=(guint64)0,
duration=(guint64)18446744073709551615;";) 0x13beeb0
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   *******
(fakesink0:sink) E (type: segment (17934), GstEventSegment,
segment=(GstSegment)"GstSegment, flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE,
rate=(double)1, applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME,
base=(guint64)0, offset=(guint64)0, start=(guint64)41711110,
stop=(guint64)18446744073709551615, time=(guint64)0,
position=(guint64)41711110, duration=(guint64)18446744073709551615;";)
0x7fb9880051f0
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   *******
(fakesink0:sink) (156 bytes, dts: 0:00:00.000000000, pts: 0:00:00.041711111,
duration: 0:00:00.041711111, offset: -1, offset_end: -1, flags: 00004040
discont tag-memory , meta: none) 0x7f58e800a400

The first GstSegment is superfluous. Ideally qtdemux should not emit it, but
since it's not followed by any GstBuffers, it's mostly harmless.

The important one is the second GstSegment. Notice start=41711110 ns, whereas
it was zero without edit lists. Buffer timestamps are still the same, but when
doing the math to get stream time (see
https://gstreamer.freedesktop.org/documentation/design/synchronisation.html#stream-time),
we get different results:

stream_time = (B.timestamp - S.start) * ABS (S.applied_rate) + S.time

Without edit lists:

stream time PTS = (41711111 ns - 0 ns) * 1 + 0 ns = 41711111 ns

stream time DTS = (0 ns - 0 ns) * 1 + 0 ns = 0 ns

With edit lists:

stream time PTS = (41711111 ns - 41711110 ns) * 1 + 0 ns = 1 ns

stream time DTS = (0 ns - 41711110 ns) * 1 + 0 ns = -41711110 ns

Notice how the movie presentation starts close to zero whereas without the edit
list there is the equivalent time interval of a frame without image.

Unfortunately, it does not start exactly at zero because of rounding errors
inherent to GStreamer design*.

* This is a consequence of applying edits by performing addition of fixed point
nanoseconds (first frame unedited track time PTS lossly converted to fixed
point in order to set GstBuffer.pts + edit start lossly converted to fixed
point in order to set GstSegment.start). Unfortunately, since GstBuffer.dts is
unsigned, qtdemux cannot handle edits by itself, they have to be applied by
GstSegment transformation from (unsigned) buffer time to stream time.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.