[gst-devel] potential for midi support in GStreamer

[Baker, Steve]

I've thought a bit about how to add midi support to GStreamer. I'm just
going to do a brain dump of my ideas since I doubt I'll be coding any of
this any time soon.

Midi could be thought of in terms of dataflow as a sparse non-constant flow
of bytes. GStreamer works best with near-constant data flow so a midi stream
would probably have to consist mostly of filler events, sent at a constant
tick-rate.

I think the following 3 elements would be most useful initially:
- midifileparse
takes midi file format data on sink pad, and produces timestamped midi data
on output. A property will specify what the tick rate would be (default to
96 ticks per beat or something). If no data exists for a given tick, it can
just send a filler event. Timestamp would be derived from the bpm property,
and the time deltas of the midi file data.

- ossmidisink
could be added to the existing oss plugin dir, sends midi data to oss midi
sequencer. Makes extensive use of GstClock to only send out data when the
buffer/event timestamp says it should.

- alsamidisink
guess what this does. don't know whether alsa's sequencer interface would be
better than its raw midi one. My money is on raw midi.

- ossmidisrc, alsamidisrc
real time midi input.

It would be nice to be able to transform midi to audio which can be further
processed in a gstreamer pipeline. Timidity might be the best hope for this,
but I don't think they have any intention of libifying it, so any timidity
plugin would have to communicate with Timidity in some out-of-process way.
Hadess claims that there is a librarified version of Timidity out there, but
I have never looked for it.

Once we have midi streams, we can start doing fun things like writing a
midi2dparams element which would map midi data to control the dynamic
parameters of other elements, but lets not get ahead of ourselves.

cheers