[gst-devel] GStreamer MIDI support (was: "bounties"/...)

[Josh Green]

On Fri, 2004-07-16 at 04:21, Jonathan LIGER wrote:
> > > It seems to me that only GstData can be routed through pipelines.
> > So the GstMidiEvent data would need to be encapsulated by GstData.
> 
> Sorry, what I meant was that there is currently only 2 classes of GstData
> (Buffer and Events) and as many GstElements only know those 2 classes, Midi
> data has to be one of those. I though for a while about adding a new type of
> event and using the GstStructure field to pass Midi events data. It
> eventually seemed not efficient at all. The last solution is using buffers,
> and then serialized events. My idea was to use a 8 byte structure close to
> raw midi data: 1 byte of flags (might need it one day), 3 bytes for the midi
> message, and 4 bytes for the Midi tick.
> 

Would it make more sense to use a union (like the ALSA sequencer) to
pass around MIDI events? I think the advantage of this is that the event
is already parsed and so some of the pitfalls of the MIDI specification
don't have to be dealt with (like running status byte, 7 bit limitation,
etc). The size would probably be greater than 8 bytes, but by using a
union it would make it efficient in space usage.

> > GStreamer MIDI could be modeled after the ALSA sequencer, since its
> pretty sweet.
> 
> What do you mean? The interface is already imposed by the GstElement class.
> 

I suppose what I was referring to was the union structure used in the
ALSA sequencer for passing around MIDI events and the method of time
stamping. This sequencer model is pretty nice, and so I was just
pointing out that it might make sense to think of creating a system that
is essentially a good encapsulation of this model.

> > A proposal was in there for control type pads, has this been
> created yet?
> 
> I don't think so.
> 

It sounds like GStreamer currently deals with strictly constant
streaming data. How does the current model of passing this data prevent
the implementation of event/control value model? Would it be desirable
to add an additional GValue based event/control network on top of the
current GStreamer data model, for things like MIDI data, GUI controls,
MIDI custom controllers, and any arbitrary GValue? An event "push" model
so to speak.

There may be some value in using the SwamiControl system that is in
Swami (http://swami.sourceforge.net), since it is exactly what I just
described.

<SwamiControlPlug>  /* if it sounds like something useful */

SwamiControl is a multi-thread safe (at least it should be, haven't
tested it thoroughly yet) network of input/output control objects with
loop prevention. For each event that is sent between controls a
structure is created that has a timestamp, origin event (if its a
chained event), the GValue of the event, an active propagation count
(how many control "pads" are currently processing the event), and a
reference count. A control can have a specific GValue type that it
accepts, in which case any value arriving at its input is
g_value_transform'd (if possible). Non value (event) controls just
receive the raw event and can process it accordingly by checking its
type. Each control has an "active" propagation list that prevents loops
(an already active event arriving again at the input is ignored), each
time this list is scanned any out dated events (active refcount is 0)
are removed.

</SwamiControlPlug>


> > Some thought needs to be put into the idea of immediate MIDI events
> versus queued (emitted at a specific time) events.
> 
> I though about a "nearly immediate" way of routing event with a loop-based
> source element polling the alsa sequencer and calling gst_pad_push each time
> an event is received. This would result in the creation of small data
> buffers. To avoid fragmentation, data could be allocated by a memory pool
> (events are constant sized, we just need to overload the default "free" and
> "copy" method fields of GstData).
> 

I'll do some reading up on the current GStreamer API, so I'm more
informed. So please do take my comments/ideas with this in mind.

Regards,
	Josh Green