[gstreamer-bugs] [Bug 564749] New: FastTagReading interface

[GStreamer (bugzilla.gnome.org)]

If you have any questions why you received this email, please see the text at
the end of this email. Replies to this email are NOT read, please see the text
at the end of this email. You can add comments to this bug at:
  http://bugzilla.gnome.org/show_bug.cgi?id=564749

  GStreamer | gstreamer (core) | Ver: HEAD CVS
           Summary: FastTagReading interface
           Product: GStreamer
           Version: HEAD CVS
          Platform: Other
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: enhancement
          Priority: Normal
         Component: gstreamer (core)
        AssignedTo: gstreamer-bugs at lists.sourceforge.net
        ReportedBy: ensonic at sonicpulse.de
         QAContact: gstreamer-bugs at lists.sourceforge.net
     GNOME version: Unspecified
   GNOME milestone: Unspecified


Fast Tag Reading
================

Components involved:

* TagReader: interface that is used for setting the desired operation for
  an element that implements it. Very simple API (3 functions).

  Location: plugins-base/gst-libs/gst/interfaces/tagreader.[ch]

* TagReader elements: Elements that implement tagreading interface and
  are able to skip data / metadata according to selected mode.

  Location: See "elements implementing tagreader interface" section below

* TagReadBin: Bin element heavily based on Decodebin2. Autoplugs elements
  that implement tagreader interface and uses fakesinks as sink elements.
  Application needs to plug a source element to this bin and put the pipeline
  in PAUSED mode. Tagreading is finished when pipeline is prerolled.

  Location: plugins-base/gst/tagread/tagreadbin.[ch]

* PlayBin2, Decodebin2, UriDecodeBin: provides "skip_metadata" property that
  can be used to speed up the media playback startup time. It sets all
  autoplugged elements that implement tagreader interface to
  GST_TAGREAD_MODE_DATA mode, a.k.a. tell them to skip all metadata parsing
  if possible.

  Location: plugins-base/gst/playback/...

* Tagread test application: A tool for measuring tag reading speed. Basically
  it supports both tagreadbin and decodebin elements in tag reading (for
  comparison purposes), but decodebin-based reading mechanism has proven
  to be unstable - it gets stuck easily.

  Location: plugins-base/test/benchmarks/tagread.c


Principle of operation
======================

* Tagreader interface defines three tag reading modes:
  GST_TAGREAD_MODE_OFF - Normal functionality, parse both metadata and data
  GST_TAGREAD_MODE_TAGS - Report only metadata and quit as soon as possible
  GST_TAGREAD_MODE_DATA - Skip metadata to increase playback startup speed

* Tagreader interface methods:
  * gst_tagreader_set_mode() is a vmethod that needs to be implemented by the
    tag reader element. TagReadBin uses it to set the tag reading mode.
  * gst_tagreader_done() can be used by tagreader elements to inform that they
    have finished reading the tags. This only needs to be used in
    GST_TAGREAD_MODE_TAGS mode.
  * gst_is_tagreading_done_message() is an utility function to check if
    GstMessage is a "tag-reading-done" message sent by some of the tagreader
    elements.

* Tagreadbin autoplugs elements in same way as decodebin2. The only difference
  in this operation is element filtering; only elements that implement
  tagreader interface are selected.

* When a demux element is found, queue2 elements are linked to its each
  srcpad to prevent preroll deadlock. Queue size is set to 1 buffers.

* When autoplugging doesn't find any more tagreader elements to be plugged,
  fakesink is added in the end of the pipeline.

* Tagreadbin keeps track of how many tagreaders have been added to the
  pipeline (n_parsers), because it wants all tagreaders to send a "done"
  message.

* Tagreadbin counts how many parsers are finished (n_done)

* Tagreadbin counts how many fakesinks are prerolled (n_asyncs_done)

* Tagreadbin works asynchronously. When it is put into PAUSED mode, it
  returns ASYNC_START. State change completes when certain condition is
  met, and ASYNC_STOP messageis then sent. This finishes prerolling and
  whole pipeline finally goes into PAUSED state.

* Tag reading process is considered finished, when ALL of the following
  conditions are met:
  * All tagreader elements MUST have finished their operation (n_done ==
n_parsers)
  * At least one fakesink has prerolled (n_asyncs_done > 0). This rule prevents
    tag reading process not to finish too early, a.k.a. when the pipeline
    is not yet totally built.



Elements implementing TagReader interface
=========================================
* Aviparse - Implements only GST_TAGREAD_MODE_TAGS mode. When tagreading mode
  is set, aviparse refuses to work in PULL mode and thus skips the
  time-consuming index parsing.

* Wavparse - Implements only GST_TAGREAD_MODE_TAGS mode. In tagreader
  mode it sends "done" message immediately after wav header has been parsed.

* Mpegaudioparse - Sends done message immediately after stream information
  has been detected and sent as a tag.

* TagDemux - base class that are used by two parser elements:
  * id3demux
  * apedemux

* Mpegstream - Because of the nature of MPEG system stream, parser needs
  to go through certain amount of data until it can be sure that it has
  found all streams from the container. Therefore mpegstream need to demux
  multiple buffers and push audio/video frames to srcpads.



Ongoing discussion & problems/issues
====================================

* Wim Taymans suggested that element could use GST_FLOW_UNEXPECTED return
  value to inform that it has finished tag reading. However, this may not work
  properly when there are multiple tagreaders in a row, because tag finding
  order cannot be guaranteed.

* One version of tagreadbin didn't use queues after demux element. This
  configuration seemed to work fine with AVI, WAV and ID3 tags, since their
  parsers are able to extract the tags before pushing any data forward.
  Therefore tag readers sent "done" message before the pipeline was even
  prerolled. However, mpegstream doesn't work this way, and it caused a
  preroll-deadlock situation; demuxer wasn't able to demux more data, 
  since audio/video stream was already prerolled and there wasn't room
  to store extra buffers. Adding the leaky queues after demux helped, but
  it also slowed down the process about 5 seconds when parsing 200 AVI clips.

* It is not good idea to upgrade any decoders to implement tag reader
  interface, since it would cause a lot of slowdown because of decoding
  process.

* The GST_TAGREAD_MODE_DATA was designed to increase playback startup speed.
  For example ID3 tag may contain album cover art, that is normally parsed
  into GstBuffer and sent to application. If app is not interested in the
  tags at this point (it may have a metadata database already) so it can
  tell Playbin2 to skip the metadata parsing entirely and just start playing
  it as quickly as possible. As you have seen above, not many components
  currently implement this.


-- 
See http://bugzilla.gnome.org/page.cgi?id=email.html for more info about why you received
this email, why you can't respond via email, how to stop receiving
emails (or reduce the number you receive), and how to contact someone
if you are having problems with the system.

You can add comments to this bug at http://bugzilla.gnome.org/show_bug.cgi?id=564749.