[gst-devel] MIDI in Gstreamer

[Andrew "Silver Blade" Greenwood]

Hi all, I'm new to Gstreamer, so if I ask something completely stupid, 
be gentle ;)

Seems like MIDI has been ignored for the most part with Gstreamer. I've 
been shown a document detailing discussions so far regarding MIDI in 
Gstreamer, and my initial ideas appear to coincide with those already 
suggested, which is good.

I'm interested in implementing MIDI as:
1) I'm writing a program which will use MIDI and audio extensively
2) My own routing API - which I've given up to use Gstreamer instead - 
was going to deal with MIDI
3) I've worked with MIDI for quite a while so know the standard fairly 
extensively

I'm also interested in implementing a "CV" (Reason-speak... Control 
Voltage/Value?) pad/data type, for the purpose of sending timestamped 
controller values between elements. This is useful for example in the 
case where you might want one element to generate values going from 0.0 
to 1.0 and back, and use those values in another element to control 
volume/panning, etc. This would look something like this:

Generator Output Pad -> Effect Input Pad

(Where both pads are in CV/controllervalue/controlvalue/whatever format)

This isn't essential, but it'd be useful for my app, and I imagine maybe 
other apps too. It might be possible to sit this on top of the MIDI 
protocol, using one of the general purpose or "effect" controllers which 
aren't assigned to a specific function. Or even have elements generate 
events/parameter changes occurring in other elements. I'm not too sure 
of this though.

So we need formats. audio/midi might not be suitable, due to audio/* 
having channel and rate properties - MIDI doesn't really have these. 
Also we need one for the MIDI file format (I assume?) and one for the 
MIDI stream. Any suggestions?

My main purpose for writing this message though is to get an idea what 
people want from MIDI.

For those of you unfamiliar with MIDI, the format generally goes 
something like this:
Status Byte - Data 1 - Data 2

Tack a timestamp on the start of that and you have a MIDI event stream, 
I guess.

Status Byte is always 0x80 or higher - this is to enable MIDI devices to 
just spit out the data bytes, and the last status byte will be assumed. 
Thus all data bytes must be under 0x80.

Status bytes between 0x80 and 0xEF have the low 4 bits used to indicate 
which channel the message refers to (0x80 - channel 1, 0x81 - channel 2 
... up to 16)

Status bytes 0xF0 and above are system messages and not 
channel-dependent. I'll not go into detail about these at the moment.

Currently, I have ideas for the following elements:
SMF_IN - Standard MIDI File input - takes a filesrc (or other input?) 
and provides a stream of MIDI events
SMF_OUT - same as above, but takes MIDI events and spits out a MIDI file
MIDI_IN - Hardware MIDI input - provides a stream of MIDI events from a 
device
MIDI_OUT - same as above, for output

MIDI_SPLIT - take 1 MIDI input - provide N MIDI outputs (repeats the 
same stream N times)
MIDI_MERGE - take N MIDI inputs - provide 1 MIDI output (to join 
multiple MIDI streams)

MIDI_DEMUXER - take 1 MIDI input - provide 17 MIDI outputs (1 for each 
channel, and 1 for system messages)
MIDI_MUXER - take 17 MIDI inputs (as above) - provide 1 MIDI output

MIDI_MESSAGE_FILTER - take 1 MIDI input - provide 7 MIDI outputs - spits 
out messagse on each pad depending on the category (note on/off, 
controller value change, patch/program select, etc.)

MIDI_SYSEX_FILTER - take 1 MIDI input - provide 1 MIDI output - removes 
messages between 0xF0 and 0xF7 (not really useful messages unless doing 
advanced things like dumping hardware equipment configurations etc.)


There's probably oh-so-much more that can be done with this, but those 
are my initial ideas - mainly for the purpose of manipulating MIDI 
messages so that they can be routed as you wish really.

Another possible element would be one that takes a MIDI input and 
provides an output where all the messages refer to a specific channel. 
This isn't really necessary as long as elements that deal with one 
channel only ignore the low 4 bits of each message...


Also, as a side-note, has anyone considered implementing elements to 
make use of PortAudio / PortMIDI? This might make Gstreamer a bit more 
portable as I'd love to use it for Windows stuff as well as Linux. Just 
a thought as it might help cut a few corners...

-Andrew