[Bug 755220] New: Feature requests for playbin/decodebin/uridecodebin and/or successors

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=755220

            Bug ID: 755220
           Summary: Feature requests for playbin/decodebin/uridecodebin
                    and/or successors
    Classification: Platform
           Product: GStreamer
           Version: git master
                OS: Linux
            Status: NEW
          Severity: enhancement
          Priority: Normal
         Component: gst-plugins-base
          Assignee: gstreamer-bugs at lists.freedesktop.org
          Reporter: dv at pseudoterminal.org
        QA Contact: gstreamer-bugs at lists.freedesktop.org
     GNOME version: ---

Hello,

these are some notes about my experiences with these bins, and what features
should be included in them, or in successor elements.


1. Proper prebuffering/preparing in playbin
Right now, the gapless playback in playbin relies on downstream elements to
have enough data buffered to cover the transition. In other words, playbin
*doesn't* prepare the next stream before the current one ends. If the
preparation isn't done very quickly, the queues downstream will eventually run
out of data. In addition, if the stream is for example h.264 data, and a
hardware h.264 decoder is to be used, an initial delay can occur, which would
cause a noticeable stutter during the transition. This can be seen on embedded
hardware like the i.MX6 devices.

I based my code on the concat element that is in GStreamer since version 1.6.
With it, such prebuffering is possible. The downside is that a lot of steps
that playbin does have to be done manually.

Prebuffering/preparing does introduce a new set of issues and necessities.
First, preparing the next stream in the background causes an increase in CPU
usage, since in that situation, *two* streams are active (though only one is
producing output). This is particularly relevant on embedded devices with
limited CPU power. It is therefore necessary to (a) prioritize the threads of
the streams (give more CPU slices to the current stream) and (b) throttle the
computations in the second stream, for example by limiting the input data rate
in the streams with a probe and a condition variable inside that probe.
Second, if both the current and the next stream receive data over the network,
then the current stream must be given priority. This can be solves in a similar
way to (b) above, by throttling the data rate in the next stream.


2. Make embedded devices first-class citizens during developments

Improvements in these bins and/or development of successors must involve
embedded hardware during development, since a lot of situations where
deadlocks, "clicks", stuttering etc. occur are not reproducible on more
powerful machines such as PCs. I have seen many of such issues on a single core
Sitara AM3352 machine with 720MHz, while the PC build showed none of these
issues.


3. Buffering improvements

"Buffering" in the bins is currently ill-defined and hardly introspectible.
Stats such as the current buffer level are inaccessible. Buffer sizes/durations
cannot be adjusted during playback. The buffering information must be a lot
more detailed and be adjustable on the fly with more fine-grained controls.
This is for example necessary with polluted Wi-Fi environments and HTTP
transmissions of VBR data such as FLAC or Ogg Vorbis. Initial buffer size
estimations can be way off with VBR, leading to frequent interruptions. A
player's buffer logic could then resize the buffer based on observations made
earlier. But right now, this is not doable with playbin/(uri)decodebin.


Comments are welcome. So are additional ideas for improvements.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.