[Bug 793058] qtdemux: add support for edit lists for fragmented files in push mode

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793058

--- Comment #14 from Alicia Boya García <aboya at igalia.com> ---
(In reply to Thiago Sousa Santos from comment #13)
> AFAIK this code is for both push and pull mode

That's my bad, qtdemux_parse_segments() is used in both modes indeed.

> AFAIK this code is for both push and pull mode to get correct PTS and DTS
> alignment. Initial PTS should be 0 if DTS could be negative but it can't so
> we shift PTS forward (together with segment.start) and keep initial DTS as 0
> to achieve the same effect.

That's not what we want or what the edit list states. Most edit lists,
including the provided test file, shift backwards, so that movie PTS (=stream
time PTS in gstreamer) starts at zero and movie DTS (=stream time DTS in
gstreamer) starts negative.

> 
> elst_shift is only non-zero on the scenarios outlined on the spec:
> 
> """
> A non‐empty edit may insert a portion of the media timeline that is not
> present in the initial movie, and
> is present only in subsequent movie fragments. Particularly in an empty
> initial movie of a fragmented
> movie file (when there are no media samples yet present), the
> segment_duration of this edit may be
> zero, whereupon the edit provides the offset from media composition time to
> movie presentation time,
> for the movie and subsequent movie fragments. It is recommended that such an
> edit be used to
> establish a presentation time of 0 for the first presented sample, when
> composition offsets are used.
> """

I think you are misunderstanding there the word "fragments" as edits. That's
not it, it actually refers to MP4 fragments (moof+mdat boxes that can extend a
moov and are usually used in DASH and MSE).

What the spec is stating:

* Edits can include frames from the fragments too. In other words, edit lists
can be used with fragmented files (like the provided example).
* Fragmented files are divided into a non-fragmented part and one or more
fragments. Fragments have different structures for the frame index than the
non-fragmented part, so usually, for simplicity and orthogonality, fragmented
files have no frames at all in the non-fragmented part, which serves only as a
header. In that case, it's useful to set the edit segment_duration to 0, which
shall be interpreted as "until the end of the complete movie, including
fragments".
* It's recommended that you use an edit list so that your movie PTS starts at
zero.

> Do you have a sample file that meets this criteria but fails? I'd be happy
> to look at it. The missing check is if the first segment is empty but that
> could be added, maybe.

I already linked you a test file above. If you demux it with your patch you'll
see PTS is being shifted forwards, instead of backwards. You can also check
that the file uses segment_duration=0 indeed, which is usual though not a
requirement (i.e. a fragmented file may specify the (non-zero) duration of the
movie with fragments in elst.segment_duration and that would still qualify as a
completely fine basic edit list).

http://yt-dash-mse-test.commondatastorage.googleapis.com/unit-tests/media/car-20120827-86.mp4

Some time ago I wrote a document about the MP4 format, fragments, edit lists
and why they are necessary. I have extended that last section and added some
notes about how they are implemented in GStreamer too. I think it would be
helpful if you gave it a look. If something does not seem to make sense, you
can ask me in IRC.

https://github.com/ntrrgc/media-notes/wiki/MP4-Notes#track-edit-list

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.