[gst-devel] media-info, and old streaminfo tags

[Martin Soto]

On Sun, 2004-03-07 at 11:09, Thomas Vander Stichele wrote:
> ...
> That's like saying "we don't really need to write down any designs for
> GStreamer, it's all in our head".  It's getting really tiring to switch
> internal design stuff around only to find out that while it fixes the
> problem we were having, it creates a whole bunch of new ones we weren't
> having.  The only way to avoid that in the future is to start writing
> down designs and prove *on paper* that apps will work this way.

Amen brother! There are many areas of GStreamer that urgently need to be
openly discussed, and just having someone hacking on them doesn't fit
the bill. Many topics are really difficult, and they raise different
issues for different people. Reaching some consensus before a solution
is implemented is absolutely necessary to guarantee at least decent
results.

For example, in the last days, I've been dealing with a lot such issues
in order to play DVDs properly. DVDs are quite demanding, because of
their interactive features. In normal DVDs, you can, and you do, find
video and audio that play synchronized, audio that plays alone while
video is frozen (a still frame, actually) and video that plays without
any background audio (I have at least one DVD with an animated menu
having no sound at all). And, of course, you have still frames used as
menu backgrounds, where you have to keep the subpicture unit playing,
while sound and video are frozen.

The good news is that, with enough hacking, you can make it work with
GStreamer. Yesterday, for example, I flawlessly played the whole Matrix
in "follow the white rabbit" mode (yeah! :-) and everything was based on
my shiny brand new components compiled against latest CVS. The bad news,
on the other hand, is that in order to achieve that, you have to hack
all over the place. Thinking of it now, I had to hack or create from the
scratch almost every single component I have in my pipeline right now. 
Only the pipeline bin and the threads are the stock ones, because even
the queues had to be modified to negotiate capabilities properly.

And the somewhat discouraging conclusion is that the whole thing works
only because I hacked everything, from dvdnavsrc, to the new dvddemux,
to the standard gstqueue to all output elements, to make it operate
consistently. So, if you have a DXR3 card connected to your TV set, and
a sound card supported by ALSA that is attached to your external Dolby
Digital decoder with an SPDIF link, then yes, you can play DVDs with
GStreamer (or at least you will be able to play them if I ever get CVS
access. ;-) If you lack any of those things, then you can probably
forget it, unless you are willing to modify whatever additional
components you need to for your own setup in potentially non trivial
ways.

At this point of time, I can not guarantee that my elements will work
properly with any other element in GStreamer. They may work, but you
never know. There seem to be a lot of reasons for that. For example,
discontinuous events are handled by different elements in slightly
different ways. They don't have defined semantics and you see it all
over the place. So, depending on your pipeline, doing something as
obvious as skipping a chapter in a DVD may result in a nice, clean
transition, or it may result in your video and audio jumping, clicking
and changing tempo like crazy, to only resynchronize a few seconds
later, if they resynchronize at all.

The clocking system, even with the recent fixes from Benjamin, has still
some serious issues. For example, as discussed this week, most audio
clocks increase time only as the providing sink plays audio material.
That means, if you have no audio, your clock stops. So, don't play any
DVDs having animated menus with no audio. Or at least, don't go into
such menus, or your GStreamer based DVD player will mysteriously freeze.

And that not to mention schedulers. At the moment, I can only say that
my pipeline works. But I'd rather not try and change it. The other day I
tried to put the video and subpicture output components in a single
thread (it should work, the spu thing has very low throughput) but the
scheduler froze. Basically, you know that schedulers work when they want
to, and only when they want to. And having many of them doesn't help,
because they all seem to be as temperamental as primadonnas.

But enough ranting. GStreamer is a really cool project and it deserves
as much success as possible. Please start discussing some of this stuff
now. It is not that hard, just write a proposal, send it to the mailing
list, wait for some feedback, correct, publish again. As soon as
something is decided, we (or at least those with CVS access ;-) can help
fixing elements until everything behaves consistently. And to put my
work, it not my money, where my mouth is, I'll start with a proposal for
clocking and discontinuity handling. You can expect it at some point
this week.

Cheers,

M. S. (who, all things considered, is still proud of watching his DVDs
with GStreamer)
-- 
Martin Soto <soto at informatik.uni-kl.de>