[gst-devel] MIDI in Gstreamer

[Andrew "Silver Blade" Greenwood]

I'd be interested in there being some MIDI plugins. In fact, I believe 
some commercial audio/MIDI apps already do this.

Is there no way to bundle MIDI into a continuous stream and have plugins 
read MIDI data from it? May seem like a stupid suggestion but I'm not 
really sure how it'd work.

The problem with using a controller-value-to-Gstreamer-event approach is 
that I don't think you'd "see" the connection between the elements. By 
that I mean a controller pad wouldn't connect directly to another 
controller pad, I think? Or am I wrong?

Maybe the element receiving controller values could set its own params?

I noticed from the mailing list archive that people appeared to be a 
little scared of /dev/midi - I took this as a bad thing, but upon 
researching, it's almost identical (if not easier to use) than the 
Windows MM system. Just fire MIDI data at it and it works. ALSA provides 
a few advanced features that some developers might want, however, but 
it'd be a case of translating MIDI events into ALSA events, which would 
then be translated back into MIDI events again.

Orrrrr... We could have a separate pad type for ALSA... Which would 
provide advanced functionality, but at the cost of sacrificing pure MIDI 
compatibility...



Stefan Kost wrote:

> Hi Andrew,
>
> for me the question is, does anyone wants to write plugins that 
> process midi data, such as a midi echo. If so we need midi as a stream 
> format. The difficulty here is that there is not a lot of code that 
> handles sparse streams. GStreamer is fine handling mostly continous 
> data, like audio and video.
>
> Personally I think playing midi or tracker music is more like setting 
> up sound-generators and effect, wire them and connect to a sound-sink. 
> Further set up GstControllers to the element properties that push in 
> the control changes from the tracks at the right time.
>
>
>
>> Hi all, I'm new to Gstreamer, so if I ask something completely 
>> stupid, be gentle ;)
>>
>> Seems like MIDI has been ignored for the most part with Gstreamer. 
>> I've been shown a document detailing discussions so far regarding 
>> MIDI in Gstreamer, and my initial ideas appear to coincide with those 
>> already suggested, which is good.
>>
>> I'm interested in implementing MIDI as:
>> 1) I'm writing a program which will use MIDI and audio extensively
>> 2) My own routing API - which I've given up to use Gstreamer instead 
>> - was going to deal with MIDI
>> 3) I've worked with MIDI for quite a while so know the standard 
>> fairly extensively
>>
>> I'm also interested in implementing a "CV" (Reason-speak... Control 
>> Voltage/Value?) pad/data type, for the purpose of sending timestamped 
>> controller values between elements. This is useful for example in the 
>> case where you might want one element to generate values going from 
>> 0.0 to 1.0 and back, and use those values in another element to 
>> control volume/panning, etc. This would look something like this:
>>
>> Generator Output Pad -> Effect Input Pad
>>
>> (Where both pads are in CV/controllervalue/controlvalue/whatever format)
>>
>> This isn't essential, but it'd be useful for my app, and I imagine 
>> maybe other apps too. It might be possible to sit this on top of the 
>> MIDI protocol, using one of the general purpose or "effect" 
>> controllers which aren't assigned to a specific function. Or even 
>> have elements generate events/parameter changes occurring in other 
>> elements. I'm not too sure of this though.
>>
>> So we need formats. audio/midi might not be suitable, due to audio/* 
>> having channel and rate properties - MIDI doesn't really have these. 
>> Also we need one for the MIDI file format (I assume?) and one for the 
>> MIDI stream. Any suggestions?
>>
>> My main purpose for writing this message though is to get an idea 
>> what people want from MIDI.
>>
>> For those of you unfamiliar with MIDI, the format generally goes 
>> something like this:
>> Status Byte - Data 1 - Data 2
>>
>> Tack a timestamp on the start of that and you have a MIDI event 
>> stream, I guess.
>>
>> Status Byte is always 0x80 or higher - this is to enable MIDI devices 
>> to just spit out the data bytes, and the last status byte will be 
>> assumed. Thus all data bytes must be under 0x80.
>>
>> Status bytes between 0x80 and 0xEF have the low 4 bits used to 
>> indicate which channel the message refers to (0x80 - channel 1, 0x81 
>> - channel 2 ... up to 16)
>>
>> Status bytes 0xF0 and above are system messages and not 
>> channel-dependent. I'll not go into detail about these at the moment.
>>
>> Currently, I have ideas for the following elements:
>> SMF_IN - Standard MIDI File input - takes a filesrc (or other input?) 
>> and provides a stream of MIDI events
>> SMF_OUT - same as above, but takes MIDI events and spits out a MIDI file
>> MIDI_IN - Hardware MIDI input - provides a stream of MIDI events from 
>> a device
>> MIDI_OUT - same as above, for output
>>
>> MIDI_SPLIT - take 1 MIDI input - provide N MIDI outputs (repeats the 
>> same stream N times)
>> MIDI_MERGE - take N MIDI inputs - provide 1 MIDI output (to join 
>> multiple MIDI streams)
>>
>> MIDI_DEMUXER - take 1 MIDI input - provide 17 MIDI outputs (1 for 
>> each channel, and 1 for system messages)
>> MIDI_MUXER - take 17 MIDI inputs (as above) - provide 1 MIDI output
>>
>> MIDI_MESSAGE_FILTER - take 1 MIDI input - provide 7 MIDI outputs - 
>> spits out messagse on each pad depending on the category (note 
>> on/off, controller value change, patch/program select, etc.)
>>
>> MIDI_SYSEX_FILTER - take 1 MIDI input - provide 1 MIDI output - 
>> removes messages between 0xF0 and 0xF7 (not really useful messages 
>> unless doing advanced things like dumping hardware equipment 
>> configurations etc.)
>>
>>
>> There's probably oh-so-much more that can be done with this, but 
>> those are my initial ideas - mainly for the purpose of manipulating 
>> MIDI messages so that they can be routed as you wish really.
>>
>> Another possible element would be one that takes a MIDI input and 
>> provides an output where all the messages refer to a specific 
>> channel. This isn't really necessary as long as elements that deal 
>> with one channel only ignore the low 4 bits of each message...
>>
>
> Having windows sinks and source for audio and video would be nice. It 
> just not yet be done. We are still waiting for takers ;)
>
>> Also, as a side-note, has anyone considered implementing elements to 
>> make use of PortAudio / PortMIDI? This might make Gstreamer a bit 
>> more portable as I'd love to use it for Windows stuff as well as 
>> Linux. Just a thought as it might help cut a few corners...
>>
>> -Andrew
>>
>
> Stefan