[gst-devel] GStreamer 0.2.0 roadmap/tasklist

[Erik Walthinsen]

On 7 May 2001, Wim Taymans wrote:

> > The main new features in the core of GStreamer in the new INCSCHED1
> > system which Erik details here:
> > http://www.geocrawler.com/lists/3/SourceForge/1504/25/5665863/
> > and Wim's new dynamic autoplug architecture described here:
> > http://www.geocrawler.com/lists/3/SourceForge/1504/0/5680778/

There are a bunch of smaller features, fixes, and enhancements as well.  A
0.1.1->0.2.0 diff should reveal them to those who know the code, but we
can start the list now:

Improved (i.e. correct) thread interlocking
Reordered states (NULL -> READY -> PAUSED -> PLAYING)
Cothread locking for cross-thread safety
Basic interruptibility across thread boundaries (not yet done)
Rudimentary -launch bash commandline completion
Rudimentary gdb debug macros

There are more things like that, please expand on the list as you think of
them.

> autoplugcache: autoplugcache: AutoplugCache
>   Works.
FYI, this is what is described in the dynamic autoplug scenario, and has
been tested.  The critical piece is the fact that it fires a signal during
its loopfunc that will almost always result in itself being removed from
the pipeline.  This will drive the development of a gtk_signal_emit()
wrapper that checks for things like a changed state or parent immediately
after return from the signal handler, and cothread_switch(0) to break out
of the iteration.

This also impacts cothread destruction and reuse.  Since the scheduler
will likely destroy the element itself in the middle of the signal
handler, we need to make sure that the cothread lands on a cleanup list,
which is walked only when a new context is created.  It will trylock()
each cothread, and only if it gets the lock will it take over and reuse
that cothread stack.  This works because the signal handler runs in the
element's cothread context, which means that the cothread lock has been
acquired.  Only after the cothread_switch(0) following the signal_emit()
will that lock be freed, allowing the cothread to be reused.

> gstelements: fakesrc: Fake Source
>   Works fairly well, we eventually want it to become a
>   very powerfull src element that can generate buffers in almost any
> imaginable sequence/complexity you want. Provisions for that are in
> the code but is otherwise unimplemented.
>
> gstelements: fakesink: Fake Sink
>   Works. should eventually provide access to sinked buffers so rapid
> sink prototyping can be done at the application level.
>
> gstelements: disksrc: Disk Source
>   Works somewhat. There's a huge leak in the way we handle mmapped disk
> reads. We really should fix this and unmap appropriatly.
Basically, we mmap() the entire file, whereas we should be mmap()ing in
chunks of, say, 4MB.  I'm working on a replacement called the filesrc
(which is the correct name anyway, technically) that will do this, and be
smart about it as well.

> gstelements: disksink: Disk Sink
>   Works.
Same applies here re: mmap().  The key is that we can then provide
bufferpools to allow the pipeline to write directly to disk.

> gstelements: identity: Identity
>   Works. Can be configured as a loopbased/chain based elements and has a
> sleep time arg. Buffer handoff should generate a signal for rapid
> plugin prototyping. Uses caps and bufferpool proxying.
>
> gstelements: fdsink: Filedescriptor Sink
> gstelements: fdsrc: Disk Source
>   Both Work
>
> gstelements: multidisksrc: Multi Disk Source
>   Works somewhat (not heavily tested by myself..)
>
> gstelements: pipefilter: Pipefilter
>   Needs rework. It should be remade as a DECOUPLED element.
>
> gstelements: sinesrc: Sine-wave src
>   Works. Ties audio caps to the core elements which is probably not a
> good idea.
>
> gstelements: tee: Tee pipe fitting
>   Works. doesn't do caps nor bufferpool proxying.
>
> gstelements: httpsrc: HTTP Source
>   Works, might suffer bitrot.
It makes use of libghttp, which is a dependency we may not need, esp since
its API isn't too compatible with the way we'd like to do things.  Might
be better to just write our own http handling, simple as it is.

> parseau: parseau: .au parser
>   Works.
>
> wincodec: winenc: Windows codec image encoder
>   Used to work, probably broken.
>
> wincodec: windec: Windows codec decoder
>   Works. need a rewrite...
>
> aviencoder: aviencoder: .avi encoder
>   Probably broken.
>
> avidecoder: avidecoder: .avi parser
>   Works, but is very badly designed. Has lots of ugly hacks and needs a
> rewrite.
The key is that the codecs are implemented in the demuxer.  The
muxer/demuxer should be just that, and the decoders separate.  Any codecs
specific to AVI only should be in the same plugin though (as should
avimux and avidemux).

> audioscale: audioscale: Audio scaler
>   Works in some circumstances. suffered bitrot.
>
> v4lsrc: v4lsrc: Video (v4l) Source
>   Works somewhat. Is very bad in caps negotiation and breaks when
> looking at it. needs a rewrite.
>
> cdparanoia: cdparanoia: CD Audio (cdda) Source, Paranoia IV
>   Works pretty well. Has suffered bitrot (no updated audio caps)
>
> stereo: stereo: Stereo effect
>   bitrotten. no caps etc...
>
> volume: volume: Volume control
>   bitrotten. no caps etc..
>
> esdsink: esdsink: Esound audio sink
>   Works somewhat. doesn't use caps to set up the audio format, so it
> breaks pretty easily.
Depends on your machine.  It sounds (sounded?) horrible in some cases on
my laptop, but worked just fine on matth's desktop with the same source
file and pipeline.  Luck of the draw.

> median: median: Median effect
>   bitrotten, doesn't use caps.
>
> smooth: smooth: Smooth effect
>   bitrotten, doesn't use caps.
>
> colorspace: colorspace: Colorspace converter
>   somewhat works. Breaks very easily due to poor capsnego
> implementation.
Also has that pesky Hermes dependency.

> stereo2mono: stereo2mono: Stereo to Mono converter
>   Should work.
>
> volenv: volenv: Volume Envelope
>   Works, capsnego is badly implemented though.
>
> adder: adder: Adder
>   Works.capsnego badly implemented.
>
> icecastsend: icecastsend: An Icecast  plugin
>   Works, I think.
Has it *ever* been tested?

> jpeg: jpegenc: jpeg image encoder
>   bitrotten, capsnego sucks.
>
> jpeg: jpegdec: jpeg image decoder
>   Works somewhat.
There are some patches pending from dichro (Mikolaj) that should help.

> mpg123: mpg123: mpg123 mp3 decoder
>   Works somewhat. isn't thread safe, doesn't work on PPC.
>
> mp3parse: mp3parse: MP3 Parser
>   Works.
>
> mad: mad: mad mp3 decoder
>   Works pretty well.
>
> lame: lame: L.A.M.E. mp3 encoder
>   Works pretty well.
>
> mpeg1encoder: mpeg1encoder: mpeg1 and mpeg2 video encoder
>   Broken.
>
> mpeg_play: mpeg_play: mpeg_play mpeg1 video decoder
>   Works very well.
>
> mpegaudio: mpegaudio: mpegaudio mp3 encoder
>   Works. src pad templates are not implemented..
>
> mpeg1parse: mpeg1parse: MPEG1 Parser
>   Works pretty well.
>
> system_encode: system_encode: MPEG1 Multiplexer
>   somewhat broken.
>
> mp1videoparse: mp1videoparse: MPEG 1 Video Parser
>   Works.
>
> ac3dec: ac3dec: ac3dec mp3 decoder
>   Works.
Works as well as ac3dec ever did...  Better than current ac3dec CVS
though.  Is very quiet, which is made obvious by the fact that Tom's
Hardware made a big deal about FlasK MPEG adding some volume controls to
counter the fact that the decoded audio was always quiet.

> ac3parse: ac3parse: AC3 Parser
>   obsolete
>
> mpeg2dec: mpeg2dec: mpeg1 and mpeg2 video decoder
>   Works very well.
>
> mpeg2enc: mpeg2enc: mpeg1 and mpeg2 video encoder
>   Works somewhat. needs rewrite.
>
> mpeg2parse: mpeg2parse: MPEG System Parser
>   Works well.
>
> mpeg2subt: mpeg2subt: MPEG2 subtitle Decoder
>   bitrotten.
>
> mpeg2play: mpeg2play: mpeg1 and mpeg2 video decoder
>   bitrotten..
>
> mp2videoparse: mp2videoparse: MPEG 2 Video Parser
>   bitrotten.
>
> rtjpeg: rtjpegenc: RTjpeg encoder
> rtjpeg: rtjpegdec: RTjpeg decoder
>   both broken.
>
> vcdsrc: vcdsrc: VCD Source
>   bitrotten, doesn't do caps.
>
> videoscale: videoscale: Video scaler
>   somewhat works.
Should this be in the same plugin as the colorspace converter, so that
they can cooperate to optimize in some cases (like do scaling during color
conversion, or vice versa)?

> smoothwave: smoothwave: Smooth waveform
>   bitrotten.
>
> spectrum: spectrum: Spectrum analyzer
>   bitrotten.
>
> synaesthesia: synaesthesia: Synaesthesia display
>   bitrotten. doesn't work as advertised.
>
> vumeter: vumeter: VU Meter
>   bitrotten.
>
> vorbis: vorbisenc: Ogg Vorbis encoder
>   bitrotten, need to adjust to caps.
>
> vorbis: vorbisdec: Ogg Vorbis decoder
>   Works.
>
> parsewav: parsewav: .wav parser
>   bitrotten.
>
> alaw: alawencode: PCM to A Law conversion
>   Works.
>
> alaw: alawdecode: A Law to PCM conversion
>   Works.
>
> gst1394: dv1394src: Firewire (1394) DV Source
>   Somewhat works, needs more work for PAL camaras
>
> artsdsink: artsdsink: aRtsd audio sink
>   Works.
>
> dvdec: dvdec: DV (smpte314) decoder plugin
>   Works.
Much work to be done to utilize libdv better, and some work is required in
libdv anyway.

> flxdec: flxdec: FLX Decoder
>   Works. doesn't work in gstplay because capsnego issues.
>
> mulaw: mulawencode: PCM to Mu Law conversion
> mulaw: mulawdecode: Mu Law to PCM conversion
>   Both work.
I wonder if the law elements should be moved into a single plugin and
renamed?  mulawtopcm, pcmtomulaw, alawtopcm, pcmtoalaw, mulawtoalaw,
alawtomulaw.  Since they share the same LUT-based code, they should
practically all be the same element, with just a table-selector argument.
Thoughts zaheer?

> ossaudio: osssink: Audio Sink (OSS)
>   Works pretty well. Could do with some more caps checking.
>
> ossaudio: osssrc: Audio Source (OSS)
>   bitrotten, needs caps stuff..
>
> ossaudio: ossgst: Audio Wrapper (OSS)
>   somewhat works.
>
> intfloatconvert: int2float: Integer to Float effect
> intfloatconvert: float2int: Float to Integer effect
>   status unknown. should work by looking at the code.
>
> rtpsend: rtpsend: RTP plugin
>   Should work.
Plugin should be renamed rtp, and rtpreceive written.

> xvideosink: xvideosink: Video sink
>   Somewhat works. Could do with some more caps checking.
Also needs to be set up so lack of various XFree extensions doesn't kill
it completely.

> aasink: aasink: Video sink
>   Works pretty well.
>
> ALSA
>   unknown
>
> LADSPA
>   Doesn't work

> Yup, that's why I'm going to get mpeg2enc libraryfied and use that as a
> base for a good, high quality encoder plugin.
The mpeg2enc author is very excited about:

1) librarification if mpeg2enc
2) using gstreamer for all the mjpeg/lav-tools stuff
3) using libcodec for optimization

> Yup, it needs a rewrite. People with interestin GUI apps: this is a
> chance to create something really cool. Takers will have great indept
> knowledge of the gstreamer core and object model.
There's a project called Seashore that should completely replace the
graphing side of the editor.  I caught a glimpse of it at GUADEC, and the
agrees that it would be ideal.  It's not released yet (due in a month
maybe?), but I'm gonna email the author and see if we can get a preview
release ;-)

> error recovery is crucial here. It needs a solid support from the core
> though, this is why it breaks sometimes.
Yup.  Being more verbose when a plugin doesn't exist, including the core
ones like colorspace ;-)  The autoplugger has a lot to do with this, of
course.


      Erik Walthinsen <omega at temple-baptist.com> - System Administrator
        __
       /  \                GStreamer - The only way to stream!
      |    | M E G A        ***** http://gstreamer.net/ *****
      _\  /_