[gst-devel] Re: Distributed pipelines and modular synthesis ramble

[Erik Walthinsen]

> I'm new to this list and in fact to gstreamer as a whole. I found my way
> over here from Slashdot as I suddenly became convinced gstreamer offers
> the best stream processing architecture I've seen on Linux so far. In
> particular I like the seperation of filters from the sinks and sources.
Yeah, that's the main reason I think it's superior to everything else.
Even DirectShow has that distinction...

> One thing that occured to me was that there are two big benefits of such
> a pipeline architecture one of which has been thoroughly explored,
> plugins, but the other I havn't seen much mention of. That is the
> ability to distribute components in the pipeline to a network of
> machines beyond a simple client-server model.
Yeah, we haven't played much with the network yet, since we've been
focusing on the core and some of the plugins.  There is a httpsrc, as well
as a shoutcast sender (untested), but we do need someone to give some more
network plugins (like a shoutcast receiver with some clue) a try.

> The reason I see this as a useful feature is because, as far as I can
> tell,  the gstreamer architecture would lend itself perfectly to
> real-time modular synthesis e.g. Reaktor ( http://www.native-
> instruments.de/english/2_products/1_reaktor/1_reaktor.html )whereby
> audio signal generators (sources) are connected to a network of filters
> and out through an audio channel (sink). One of the limitations I've
> found using Reaktor extensively is that when I have a couple of
> synthesizers running simultaneously alongside a sequencer, my machine
> practically freezes. I've started thinking recently about the
> posibility of making a distributed modular synthesizer so I can put the
> processor intensive parts on some other machine(s) that doesn't affect
> the rest of the applications running.
Yeah, I've been thinking along those lines from the very beginning of the
project actually.  That's why RidgeRun was so impressed the hired me on
the spot ;-)  Basically, I'm into live mixing of audio, with many
channels.  This works as long as you don't try to put effects on too many
channels.  Beyond a certain point, you're going to need separate hardware,
which is where I'd want to get my hands on a firewire-connected DSP farm.
The ability to move big chunks of the pipeline over to the more dedicated
machine is critical for that, and GStreamer is designed to be flexible
enough for that to happen easily.

At RidgeRun we're going to be working on (even starting this week maybe)
the basis for this stuff, which consists of a Bin that uses ORBit
internally to communicate with another process[or].  Our use for it is to
move parts of the pipeline to a 'conjoined' DSP, since we're working with
TI on chips that have an ARM CPU and a DSP on the same die(chip).  The
goal is to be able to control the whole pipeline from the GPP
(general-purpose processor, the ARM) transparently, letting GStreamer and
this specialized Bin do the work of network transparency for all
non-dataflow operations.  Dataflow operations would probably also be
handled by the Bin.

Some of this will be closed source, simply because we (RidgeRun) have no
choice when using TI's software.  But I will do everything I can to make
a generic GPP-level Bin, which we can subclass for DSP stuff just as
easily as we can subclass for network transparency.  The trick is to make
it efficient.  But I'm getting good at that, so.... <g>

> In fact, maybe you guys are sitting on a revolution in music production
> as a whole whereby the traversal of sound processing from dedicated
> hardware to computer software could become complete with just a few PCs
> sitting around the studio with gstreamer pipelines doing all the number
> crunching.
Exactly.  Wim Taymans has been thinking along those lines from the video
point of view.  A large farm of inexpensive machines could crank through
video processing in realtime, and you could even make use of it with a
fast/wide enough network.

> Has this been brought up before? Would this be a viable application for
> gstreamer? Or should I go back on the medication?
Meds are good, but yes, this stuff has been on our minds for a long time.
If you're interested in helping out, we could use all the help we can get
to bring GStreamer into these areas.

      Erik Walthinsen <omega at temple-baptist.com> - System Administrator
        __
       /  \                GStreamer - The only way to stream!
      |    | M E G A        ***** http://gstreamer.net/ *****
      _\  /_