[gst-devel] One Stream API to Rule them all - Could GStreamer be it ?

[Brian Crowell]

On Wed, Dec 3, 2008 at 3:18 AM, Maxim Mueller <maxim_m at gmx.net> wrote:
> automatic threading, scheduling, detection/plugging services and a generic
> interface to processing data would benefit the programming ecosystem
> greatly, imho

All right, I'm a Linux and GStreamer newb, but I most certainly have
an opinion on this. Caveat lector.

You're basically asking that there be some sort of generic streaming
API available, potentially in the kernel, that auto-does everything.

First, you have a streaming API in the kernel, and it does do
automatic threading and scheduling, and just about every command line
tool uses it. It's stdio: files and sockets. It does not do stream
typing and autoplugging, but UNIX has made it this far without those
features. Not that they would help you anyhow-- you still have to
specify parameters to your filters and commands to get them to do what
you want.


> benefits of one globally used stream processing api:
> - greatly reduced development effort, rate of bugs and maintenance

The GNU people would agree.


> - reduced learning curve when adding functionality to software, furthermore
> a more structured way of thinking about processing data is also bound to
> produce cleaner code

Over the current method? I don't think so. Adding more complexity to
the interface makes it easier to screw up. I'm still getting segfaults
from the videomixer plugin, and I'm not at all sure why, and I know
the person who authored it knows way more about GStreamer than I do.

C++ was supposed to produce cleaner code, but I've seen many obscure
bugs happen just because a copy constructor was written wrong.


> - systems theory: standardizing components interfaces  tends to lead to the
> discovery of novel ways to combine those, leading to innovative applications
>
> - improve the functionality of existing applications simply installing a new
> plugin(to a certain degree)

Again, the GNU people would agree. In fact, typing the stream would
reduce some of your flexibility. Think of some of the novel ways
people have used grep and sed.


> - users decide which implementation they want to use, all in all, it's
> his/her decision to make.
> the ability to stop having to install multitudes of http/etc protocol,
> VirtualFileSystems(KIO-slaves,GIO, PHP5's stream API ...),
> encoding(OpenSSL/GnuTLS/PGP ...), compression etc implementations just
> because the respective application developers had their favourites, would be
> a great step forward in the fight for truly free software and putting the
> user back in control of his own system.

All of these choices would still exist, and you'd have the extra
effort of having to write brand-new GStreamer plugins to support them
all.


> - by moving up one abstraction layer, just one wrapper to support other
> programming languages is required-in contrast to per library wrappers
> .

I'm not sure what you mean by this. GPLers have the power to
consolidate their libraries at any time, but they don't. Every library
has a different purpose, and not all of it makes sense together, and
not all of it can be described in terms of streaming.


> the plugin->library separation could still be kept in selected cases to
> allow for ultra-low-level operation and alternative stream processing apis,
> although for this class of algorithms it might be overkill.

Abstraction is great, right up until you run across a problem the
abstracter didn't think about. Inevitably, trying to consolidate
everything leaves you with the problem of having to separate
everything out again (or worse, copying out just the part you need).
Look at the flak Phonon is taking for trying to abstract a little too
much.

On a related note, I just discovered liboil the other day. That rocks.


> Q: where do we stand with GStreamer in terms of being able to be the root of
> that api ?
> - is it sufficiently generic ?
> - is it simple enough ?
> - is the core sufficiently light-weight and modular ?
> - are plugins sufficiently simple  to write and do they only add a
> negligible overhead by default ?
> - is the type-system powerful enough or can it be made to be ?

What you're asking to do is probably technically feasible, but I
wouldn't count on it happening. A simpler API for GStreamer would be
awesome, but judging from its popularity, I think the design pretty
much nailed the requirements.


> Q:Should gstreamer be split into a generic stream processing/plugin mgmt
> layer and a multimedia handling api layer for this ?
> everybody should use the lower layer, and be competing on the higher API
> level(xine,mplayer,directshow ... not sure if phonon is in the same level)

It kind of already is. The core plugins don't know a darned thing
about multimedia, and the API actually makes very few assumptions in
that regard.


> Q: should the core go into the linux kernel to be universally available and
> be able to serve the kernel's needs(hashing, TCP/IP layer, crypto, VFS
> etc...) ?
> can it even go there with the plugins staying in userland ?

No. Using GStreamer for kernel tasks would only slow the kernel down.
Just the work that goes into stream type negotiation would be a waste
in many places, especially since most (if not all) of the types are
known in advance.

On the other hand, when you look at sockets and pipes, you've already
got that clean separation with all the plugins living in userland. :P


> Q: and most important of all: does all that even make sense ?

That question should have come first :P

I don't think so: you're better off with ordinary kernel streams
(stdio, sockets, pipes, etc.).

--Brian