[gst-devel] (fwd) MIDI and GStreamer

[Andy Wingo]

A forward, because somehow exim lost this before. Gah!

----- Forwarded message from Andy Wingo <wingo at pobox.com> -----

> From: Andy Wingo <wingo at pobox.com>
> Subject: MIDI and GStreamer
> To: gstreamer-devel at lists.sourceforge.net
> Mail-Followup-To: gstreamer-devel at lists.sourceforge.net
> 
> Again, apologies wrt digest-mail and mutt... and why is it that I' only
> replying to leif's mails? ;)
> 
> > Date: Mon, 14 Jul 2003 15:56:02 -0400
> > From: Leif Johnson <leif at ambient.2y.net>
> > To: gstreamer-devel at lists.sourceforge.net
> > Subject: Re: [gst-devel] Midi and GStreamer
> 
> > It seems like GStreamer could benefit greatly from a different subclass
> > of GstPad, something like GstControlPad. Pads of this type could contain
> > control data like parameters for oscillators/filters, MIDI events, text
> > information for subtitles, etc. The defining characteristic of this type
> > of data is that it operates at a much lower sample rate than the
> > multimedia data that GStreamer currently handles.
> 
> This is essentially what I do in soundscrape,
> http://ambient.2y.net/soundscrape/. The concern for a modular
> synthesizer is that anything can be hooked up to a "port" (I'll use that
> word in order to avoid prejudicing the brain with "pad") -- a
> control-rate source, an audio-rate source, or even just a scalar value.
> The element should operate with maximum efficiency in all cases. In the
> last case, you need something like port_set_value (), and the element
> checks to see if the port is connected before pulling data. The
> processing function has different algorithms based on the kind of data.
> 
> Well, why implement "ports" with pads? For the sake of interface
> simplicity, I think. I could be wrong. DParams could be the right thing,
> but they seem complicated. So for the meantime I hack around it with a
> kind of control buffer, which has the buffer data pointing to the pad's
> value (another consistency of interface) and a special flag set on the
> buffer (in addition to GST_BUFFER_DONTFREE, of course). I think that
> makes a lot more sense than playing with timestamps.
> 
> In the most efficient case, which I'll implement at some time in
> soundscrape, the chain or loop function will be set dynamically based on
> the number and kind of ports as well as what is connected (caught via
> the ::connected signal).
> 
> Which gets back to MIDI. Running the risk of ignorant philosophical
> nothing-buttery, MIDI is nothing but a representation of control
> signals. So all you need are elements to convert that representation to
> control signals. In addition, you'd probably want something like
> SuperCollider's Voicer element -- see
> http://www.audiosynth.com/schtmldocs/Help/Unit_Generators/Spawners/Voicer.help.html
> for more information on that.
> 
> All of this is pretty specific to a synthesizer system, and rightly so
> multiple projects use it it could go in some kind of library or
> what-what (that's namibian english ;) but otherwise it can stay in
> individial projects.
> 
> For some reason, I just don't care right now about dparams. Should I,
> Steve?
> 
> > Elements that have control pads could also have standard GstPads, and
> > I'd imagine there would need to be some scheduler modifications to
> > enable the lower processing demands of control pads.
> 
> Not really, the scheduler doesn't care what kind of data shuttling back
> and forth.
> 
> I'd like to end this email with a message to Nick: Nick, I don't think
> it makes sense to package amSynth(e) like that. I mean, why have two
> signal-flow graphs operating in the same process? It doesn't make sense,
> at least to me. Synthesizer modules are equivalent to GStreamer
> elements, with some customizations. Why not base a synthesizer on
> GStreamer? Of course, that's the path I chose, so I might be blind to
> other considerations. Also, your code works now with MIDI and such while
> I still need to add it to mine. But the point remains, synthesizer
> modules are the same as elements -- why have two levels of the same
> thing? What are the reasons?
> 
> Well, I've got a lot of other emails to plough through. Wait until
> gst-guile and a gstreamer 0.7 package are released and try soundscrape
> sometime, I'd also like feedback there.
> 
> Regards,
> 
> wingo.

----- End forwarded message -----