[gst-devel] GStreamer todo list

[Erik Walthinsen]

On Fri, 9 Feb 2001, Erik Walthinsen wrote:

> To do list for GStreamer
> ------------------------
> Highest Priority
> ----------------
> 1. cleanup build process, cvs branch strategy
>   DONE: seems good enough for now (thx, Omega)
There's a page up at http://gstreamer.net/cvs.shtml that has some of this.
It's not up to date, so don't take it as gospel yet.  I've got another doc
that has the correct proceedures, I'll merge that in shortly.

> 2. seeking: this changes several things
First a quick explanation of this: the current seeking is an absolute file
position.  This doesn't allow time-aware seeking.  To do this, we have to
have each element that is aware of the media type (mp3parse, mpeg2parse,
etc.) that will understand time.  You tell it what time to seek to, and it
internally makes a conversion to bytes, if it can.  It then calls
getregion on its supplier with the byte offset.

>   - add "get region"
getregion is in place, but may need some changes.  For instance, it only
works when you have loop function that has a getregion function.  If you
end up with anything between the parser and the actual source that can't
handle getregion, the seek gets lost.  So the question is: do we provide
default getregion functions that just proxy the request automatically, or
do we punt and require that anyone putting something in the middle knows
enough to put the right kind of elements there?

Another option entirely is to go out of band and tell each parser of its
left-hand peer that has seek capabilities, completely bypassing anyone in
the middle.  This has sync problems, for instance if you have a queue in
the middle.  You'd end up with the whole contents of the pipeline flushing
out before the buffer from the new seek point finally comes in.

>   - elements can set their state to "don't schedule me" for the scheduler
I think this should be implemented as a side effect of element state
change.  Whenever an element enters or leaves the PLAYING state, it tells
the scheduler to add or remove itself from the list of elements to
schedule.  This requires incremental scheduling (it *is* incsched), of
course.

>   - scheduler always schedules "the parser" first
This is necessary to ensure that the getregion is the first thing that
gets run in any given iteration.  This might force an extra bit of code in
the scheduler to make sure that each chain contains only one parser, as
defined by the CAN_SEEK bit or something.

>   - all elements between "the parser" and the source need to implement
>     "get region"... if a buffer element gets its "getregion" called, it
>     needs to flush what it has and call "getregion" to its left...
This is the core of the seeking stuff.  A seek causes a left-ward
traversal of the new seeking information.

>   - an application needs to set the pipeline to "paused" or "stopped"
>     before seeking
This is a potential problem because there are really two kinds of seeking.
The first is app-directed, the other is a function of the plugins
themselves.  App-directed seeking should be done in paused or stopped, but
if the plugin needs to seek (for instance if it's striding through data),
we have to support seeking on a live pipeline.

>   - a new state "empty" (like a partial eos) will be added so scheduler
>     can empty everything in the pipeline
This will likely consist of some variation of the right-ward cascading
state transition that EOS uses right now.  As each element gets the end of
flow marker (either EOS or empty), it would shut itself off.  The trick is
to do this in a way that doesn't mess up the scheduling, since turning off
a single element in a chain removes the entire chain from the schedule.

This makes it more interesting to have a NULL buffer pass through the
system.  But if there's some way we could abstract the EOS mechanism
instead, that would be better, to avoid the overhead of checking for NULL
buffers left and right in normal dataflow.

>   - all parsers will have to be modified to support this
Yup.

>   - all src's need to be examined... any necessary changes?
vcdsrc, dvdsrc, and cdparanoia need to be modified to deal with getregion.
The rest (fdsrc, httpsrc, etc.) can't seek anyway.

> 3. make eos more robust
>   - works fairly well now
>   - `seeking' changes will effectively change/improve this
One hopes ;-)

> 4. add error handling
>   - need to have an error model, and associated guidelines / policies
>     when elements (and parsers!) generate errors
There's some of this, but there are different classes of errors.  There
are fatal errors where the only thing left to do is stick your head into a
NULL pointer, there are errors caused by the application writer, there are
errors caused by the stream, etc.

>   - walk through several typical data paths, verify err handling for every
>     possible error condition!
>
> 5. clean up documentation
>   - changes to API => changes to docs
Hopefully we can keep that up to date with the code changes, but most
likely we're going to have to do a pass or two just before release anyway.

> 6. change plugins to be `loop func's'
>   - this is a consequence of `seeking' changes
We don't need to make them all loop functions, but we do need to
characterize what kinds of elements do need to be loop functions.  All the
parsers need to be, else they can't implement getregion (at least without
some major trickery).

> 7. create a non-GTK application to use GStreamer
>   - audio only
We have lots of cmdline programs, so this shouldn't be too hard, unless
the main loop of the application can't be attached to.  Then we have to
use a Thread to contain the whole pipeline, then we have locking and other
issues to worry about.

> 8. walk through state transitions within all elements
>   - verify appropriate behavior
>   - `seeking' changes will basically require this
I'm going through the Thread code right now and rewriting it to make it
more sane.  Basically (related to above), we have to make sure that all
the various pieces of code run in the right places.  I'm thinking that we
need to ensure that *all* the code for any given plugin always runs in the
same context.  This is especially important for elements that talk outside
their process, like pipefilter (though pipefilter should be decoupled, so
it has other work to do).  This may mean proxies on all the functions that
one normally calls on an element, for the Thread to trap.

> 9. per-pipeline memory cleanup
>   - ability to `delete' a pipeline (or any element)?
This is a matter of implementing it, but we also have to worry about the
effects of this to a pipeline in terms of scheduling and such.  Nothing
hard, though.

> Opportunistic
> -------------
>
> 10. create a voice codec (gsm?) plugin
>
> 11. naming:
>   - ensure assigned "names" are unique within context!
>   - naming convention?
We have no way of guaranteeing globally unique names, so we have to
guarantee locally unique names at a minimum.  This is one of the reasons
that the ghostpad stuff was redone the way it was, so the ghostpads
derived from different child pads can be renamed in the process.

> 12. ensure cleanup/garbage collection
>   - GtkObject: fix _destroy()
>
> 13. buffers: "metadata" needs complete rewrite
Should be based on properties, which means a complete API for fine-grained
properties manipuation (wtay is working on that for caps negociation).

> 14. buffers: add check s.t. bufpen's cannot be unknowingly overwritten
>   - at least generate some error
Some of that done, a few more places that need it.

> 15. GtkObject: ref counting not used... remove?
Probably should use it.  That's the best way to deal with destruction.

> 16. add ability to read / report ID3 header information
>   - mp3parse, etc
Do we use libid3 or not?  libid3 has lots of writing support, which we
don't need, but theoretically makes reading them a lot easier too.

      Erik Walthinsen <omega at temple-baptist.com> - System Administrator
        __
       /  \                GStreamer - The only way to stream!
      |    | M E G A        ***** http://gstreamer.net/ *****
      _\  /_