[Bug 791858] New: GstTocSetter: proposal to help standardize elements behaviour

Thu Dec 21 19:41:17 UTC 2017

https://bugzilla.gnome.org/show_bug.cgi?id=791858

            Bug ID: 791858
           Summary: GstTocSetter: proposal to help standardize elements
                    behaviour
    Classification: Platform
           Product: GStreamer
           Version: git master
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: Normal
         Component: gstreamer (core)
          Assignee: gstreamer-bugs at lists.freedesktop.org
          Reporter: fengalin at free.fr
        QA Contact: gstreamer-bugs at lists.freedesktop.org
     GNOME version: ---

TOC Setter handling needs improvements in order to ease TOC related application
developement and to standardize the behaviour of Elements that implement the
GstTocSetter interface. This is a continuation of the discussion started in (0)
and an attempt to trace and formalize what has been considered so far. This
could also serve as guidelines for Elements that implement GstTocSetter.

Please correct me where I'm wrong and review my proposal for a simple
enhancement. My idea is to extract parts of this description for the
documentation of the solution.

Context
-------

A Table Of Contents (1) is a tree structure which allows representing
subdivisions of a media. Each TOC Entry is defined by its start and end
positions, what it refers to (a chapter, an alternative angle, ...) and
optional tags such as the title, the artist... An application can select a TOC
Entry to reach the particular subdivision or aternative stream it refers to.

TOCs can be Global to a source or relative (Current) to the currently playing
stream. An application can listen to the message bus of the Pipeline and get
TOC messages (2).

>From an Element's perspective, we can distinguish two main types for TOCs
depending on their origin:

  - Upstream TOCs are received on sink pads via TOC Events (3).
  - User defined TOCs are assigned to the Element using the GstTocSetter
interface (4). The User defined TOC should persist unaltered unless the user
replaces it, resets it or if the Element is set back to NULL.

TOC Events also indicate an "updated" status which may be used by Elements to
decide what to do with the received TOC.

Some media formats may not support all the features of the TOC model. It may be
the case for alternative angles or the ability to define a Current TOC. An
Element responsible for encoding to a format may need to adapt the upstream TOC
to the format's capabilities. Moreover, there are multiple ways of representing
a TOC structure. The Element should be flexible enough to map the input TOC's
structure to fit its format's constraints.

Use Cases
---------

The following use cases are identified for an Element which implements the
GstTocSetter interface. We can consider a multiplexer such as the Matroska
Multiplexer (muxer). This muxer is responsible for encoding a Matroska
compliant output stream from multiple audio, video or subtitles source streams.
Each source stream may send a TOC. The application requirements can be splitted
into the following modes:

  - Use the Upstream TOC as the Output TOC. This is the usual behaviour when an
application doesn't need to modify the TOC. Note the following variations which
might require specific processing:

    - One Global Upstream TOC is received. The Element only needs to adapt the
TOC to fit the format's capabilities.
    - One Current Upstream TOC is received. If the Element doesn't support this
kind of TOC, it may need to merge it into a Global TOC.
    - Multiple Global / Current Upstream TOCs are received, possibly on
different sink pads. How the Output TOC should be built is context specific. An
Element may favor a particular sink pad, ignore subsequent TOC Events unless
they are marked as "updated" or perform some sort of conflict resolution.

  - Use the User TOC as the Output TOC. In this mode, the Element must ignore
Upstream TOCs and use the User TOC.
    - Just like for the Upstream TOC, the Element might implement specific
processing to match the format's capabilities.
    - The application may need to use the Upstream TOC(s) to build the User
TOC.

  - No Output TOC. The element must not output any TOC whatever the Upstream
TOC(s) received or the User defined TOC.

Current Status
--------------

GstTocSetter allows storing a user TOC and resetting the stored TOC. Each
GstTocSetter implementer handles the use cases in its own way. E.g.:

  - wavenc & flacenc use a dedicated field for the Upstream TOC. When the
output stream is encoded, the User TOC is selected only if no Upstream TOC has
been received. The user can't prevent an Upstream TOC from being encoded.
  - matroskamux stores the Upstream TOC in the GstTocSetter interface. In the
proper state, the user can access the Upstream TOC, apply any modification or
reset it. However, the User TOC can't persist unaltered.

As a workaround it is possible to define a sink pad probe (5) with
GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM, then filter TOC events and return
GST_PAD_PROBE_DROP to prevent the Upstream TOC from reaching the Element. The
application can form a User TOC or do nothing if no TOC should be written.
Another solution is to build a new TOC in the pad probe and replace the event
with an event containing the new TOC.

Proposal
--------

The proposal for this enhancement is to introduce a "selection_mode" field (and
the associated getter and setter), a GstTocSelectionMode enum with the
following possible values:

  - GST_TOC_SELECTION_UPSTREAM. This is the default when the GstTocSetter is
created and the selection_mode will be set back to this upon reset. Elements
should use the Upstream TOC if any and shouldn't encode any TOC otherwise.
  - GST_TOC_SELECTION_USER. The selection_mode switches to this value when a
TOC is set on the TOC Setter. Elements should use the TOC Setter's TOC if any
and shouldn't encode any TOC otherwise.
  - GST_TOC_SELECTION_NONE. In this mode, Elements shouldn't encode any TOC.

The workaround described in "Current Status" could be solution for applications
which require to process the Upstream TOC(s) on the fly.

---

(0) https://bugzilla.gnome.org/show_bug.cgi?id=791736
(1)
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstToc.html
(2)
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstMessage.html#gst-message-new-toc
(3)
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstEvent.html#gst-event-new-toc
(4)
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstTocSetter.html
(5)
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstPad.html#gst-pad-add-probe

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.