[gst-devel] Stuff that needs doing...

[Erik Walthinsen]

OK, now that 0.1.1 is out, it's time to figure out what we're doing next.
There are a lot of things that come to mind, so I'll just list them in the
order they pop into my head.

1) CORBA-ization
There are quite a number of places and ways to add CORBA to GStreamer:

a) Bonobo wrapper of libgstplay - already significant work has been done,
more is needed.  We need to have a standard IDL for embedded media
players, which should be run past all the GNOME people, and probably the
KDE people as well.

b) CORBA layer on top of GStreamer, allowing for easy "language bindings".
There are two ways to do this, not sure which is best yet.  One is a
single object that provides all the interfaces, and basically is a
GstPipeline.  The other would be to export all the elements individually
and figure out how that works...

c) CORBA inside a container, used as an inter-process[or] link.  This is
pretty self-contained, but runs into the problem of finding the best way
to proxy the Element interface.  If one does a set_arg on an element, how
does this data get passed to the real element living on the other side?

d) CORBA-ization of everything, in the form of making the public API for
GStreamer fully wrapped with ORBit.  This has the potential advantage that
objects are more easily movable, but that depends on how ORBit implements
it.  More looking at the guts of ORBit is necessary to see if that solves
enough problems.  It requires significant work to re-write all the
namespaces, and it could make direct use of the API rather hard.

2) Video rework
As taaz just found out, it's not clear how to do several things as far as
video goes.  We need to standardize on properties for raw video, and
make sure that everyone pays attention to them.  For instance, while
discussing a multi-camera security system, I realized that a video tiler
would be a null plugin with some bufferpool manipulations.  It would get a
single frame buffer from the videosink, then create N very similar buffers
with a slightly different offset for each, but retaining the same stride.
Then any decoder would write with the right stride into the larger frame.

This also means working out the rest of the bufferpool API.  For video, we
need to be able to ask for buffers in an out-of-order manner in some
cases, specifically MPEG video.  There may be cases where this matters to
the bufferpool provider, so there should be a way of asking for a specific
frame number or audio timestamp.

3) TimeCache
"TimeCache" is the current name for an object that brentb and I have
worked out, that manages the mapping between time and offset (or
potentially any other mapping) for an element, usually a parser.  It
stores the various points internally, and has the concept of accuracy for
both A and B values.  Data points are added in groups, which means that
eventually a whole series of points that are inaccurate can be adjusted
later to correct for guess drift.  I'll write up a prototype .h for this
soon.

4) Event system
This is the big one.  I realized a week ago that there are more things
that may be of interest to a plugin than just what comes in on its pads.
Specifically, consider an mp3parse element and a file with an id3v2 tag.
The tag is at the end of the file.  In order for the parser to seek to the
end, read the tag, and seek back, it has to know that it's got a new file.
How does it do that?  It could guess when it gets an offset 0 buffer, but
what if you start out in the middle?

The solution is some kind of event system that encompasses EOS, empty,
flush, seeking, new-file notification, and everything else.  This system
would have to be carefully constructed to follow the dataflow left to
right, and have the "right" behavoir when going right to left (seek).
Some have suggested attaching these events to the buffers, but that
introduces the overhead of checking each and every buffer at every point
in the pipeline, which we've intentionally avoided so far.

In general, a plugin would have a standard handler that would simply pass
events through.  Various plugins would either override that handler or use
some other mechanism to register for interesting events, and handle them
that way.

We need to discuss the format of these events, which events we're going to
start with, and how the plugins would interact with them.  This is
currently my second-highest priority, behind figuring out a first pass at
the cross-process[or] stuff.

      Erik Walthinsen <omega at temple-baptist.com> - System Administrator
        __
       /  \                GStreamer - The only way to stream!
      |    | M E G A        ***** http://gstreamer.net/ *****
      _\  /_