[gst-devel] media-info, and old streaminfo tags

[Martin Soto]

Hi Benjamin:

On Wed, 2004-03-10 at 15:57, Benjamin Otte wrote:
> Quoting Martin Soto <soto at informatik.uni-kl.de>:
> 
> > Well, as I explained, I'm usually sending sound, so I really need and
> > audio sink. It is just that every now and then there are passages
> > without sound, and I have to handle them as well. My older player tries
> > to modify the pipeline in such cases to disconnect the audio component,
> > and believe me, doing that raises more issues than it solves. The main
> > problem is that it is very hard to touch the pipeline without disrupting
> > the data flow. You unlink a pad a bit too early, and you may prevent
> > some important data from reaching its intended destination. This may
> > result in all sorts of funny behavior, specially in DVD menus.
> > 
> I think the correct behaviour would be to send silence in that case. Or invent 
> some sort of event to tell the stream there's nothing there. But just not 
> sending anything doesn't work. (We've got to solve the same problem for MIDI 
> btw)

I tried the silence solution already. I put a silence element in the
pipeline and tried to switch the sound to it by modifying the pipeline
when necessary. It never worked, not only because of the typical issues
with changing the pipeline, but because the scheduler never liked the
idle silence element. The event solution may work, provided we properly
define what elements should do when such an event arrives.

> > My point is that some areas of GStreamer need to be specified that way. 
> > The beauty of GStreamer is that you can mix elements freely, but in
> > practice, you end up very often with a non-working pipeline, because of
> > small inconsistencies between the various components.
> > 
> The GEP looks like it never took off. Debian and FDO are more or less 
> standards for interoperability between different software packages. And they 
> take a long time. I'm not sure you want such a thing inside a single project.

Admittedly, there hasn't been a lot of activity recently with the GEPSs,
but the other examples are alive and well. And speaking of
interoperability, that's exactly what concerns us. We need to guarantee
that plugins can interoperate regardless of their origin or authorship.
Right now, this is not the case.

> > I agree, David did a great work with the caps negotiation subsystem. And
> > one of the good things he did was putting together a text document
> > explaining how the whole thing works. You certainly know what to expect
> > of capsnego one you use it.
> > 
> If that's enough documentation for you, then I'm fine with writing that much 
> myself for the stuff that I do. I don't consider those two files in 
> docs/random more than an overview. 

Not necessarily enough, but it's a good first step.

> I've been thinking about how much documentation we need and want quite a bit 
> since this topic has come up.
> I'd divide developers using GStreamer into three sections:
> - application developers
> Those guys need an API to plug their pipelines together that is well 
> documented. If something doesn't work as expected, they file a bug and we take 
> care of it.
> examples: Ross, Colin
> equivalent (in Gtk): app developers doing their app with Gtk. They use the API 
> docs.
> - plugin developers
> Those guys need a deeper understanding of how GStreamer works internally. They 
> need design overview documents (like the PWG) and an even bigger API reference 
> for the plugin API. Depending on how hard their problem is, they might even 
> need to look into code of other plugins or the core to figure out how that 
> works.
> examples: Christophe, Julien
> equivalent: people writing their own widgets. I've had to read quite some code 
> to get my fullscreen widget do correct focus handling...
> - core developers
> They implement the new stuff. They read the code of the core and know most of 
> it inside out. Nonetheless they're happy to have some structural overview of 
> subsystems if they need to get into it and weren't before.
> examples: David, me
> equivalent: gtk developers

I agree with your classification. And it is clear that documentation
will turn scarce as you try to go deeper in the code tree. However, I
thing that at least the plugin api should be well defined. It must be
possible for third parties to develop plugins for GStreamer in a
reasonable way, where "reasonable" doesn't mean getting acquainted with
half of the code of GStreamer and hacking your way around
inconsistencies as you find them.

> All of those have quite the same documentation available as the Gtk peers. The 
> difference in most cases is that in GStreamer the stuff doesn't work 
> predictable while in Gtk it does. If your app would just disconnect the 
> elements when you didn't need them you wouldn't have thought about clocking or 
> scheduling at all.
> You probably never thought about the way Gtk allocates space to its widgets 
> and you probably don't care. Because it just works.
> 
> I'm not saying we're perfect. I'm just not thinking we're doing something 
> terribly wrong wrt documentation.

I'm not saying that either. Gtk is a much more mature project, and it
still has problems with its documentation, you cannot expect anything
else for GStreamer. I'm not trying to criticize people for what they've
done so far. I'm just pointing out that if we want GStreamer to be
widely accepted, we need to improve our internal standards. And I speak
of "we" here, because I feel I'm part of it. I've been hacking my DVD
playing stuff for almost a year now, and whenever I've had a problem,
I've been satisfied with just finding a workaround that addresses my
immediate needs. I'm now convinced that there are problems in GStreamer
that need a more global approach. What I'm advocating is that we all
(application, plugin and core developers) discuss such issues openly and
find a solution that is good enough for us all, instead of hacking local
solutions in a solo virtuoso style.

M. S.
-- 
Martin Soto <soto at informatik.uni-kl.de>