[gst-devel] proposal for implementing dynamic parameters

[Owen Fraser-Green]

Hi,

> If I might be presumptuous here, I think the different philisophical
> attitudes in this situation are:
> 1. We have a powerful abstraction of data flow (pads, data flowing from
> element to element) so lets use it wherever it makes sense.  We could also
> use it for flow of control data and (taken to extreme) other meta
> information like events and clock ticks.
> 2. Each feature of a multimedia framework has its own unique requirements
> which would be better served by tailoring an API to meet those 
requirements
> instead of overloading existing APIs which are already quite complex.

Yup, I'm with you there. I don't think low-level events and clock ticks 
lend themselves well to being integrated into the existing data flow but 
anything which can be appropriately modelled as a continuous f(t).

I also agree that gstreamer serves a broad set of applications with their 
own requirements. Perhaps we should throw in a few examples to help 
illustrate our points.

> The problem is not only that the control rate is low - in many cases it 
will
> be non-constant and not deterministic.  I don't know enough about the
> scheduler to say what effect that would have to the complexity though.

In a full real time context the control data will always be continuous e.g. 
a fader on a mixer will always be saying that stating its position even 
when it isn't changed. Thus, the data flow will be constant and 
deterministic.

In real time playback and non-RT modes, the control data won't be 
represented in the score continuously yet it will be deterministic. This 
where interpolation comes in to transform the discrete data set into a 
continuous envelope which can be applied to the data stream in a 
convolution algorithm (a la Erik's addendum to GstDynamicParams). At this 
point, it has essentially been turned into a concentional analogue signal 
(through the gst_dparam_interpolate_float_array routine) so we're 
essentially arguing the same point.

The only difference is that I'm suggesting that instead of complicating the 
scheduler with a means of getting the parameterised control data to where 
it's needed at the right time, the parameterised data should be 
interpolated at the far left of the graph and flow in the same was as audio 
data. To reiterate, all such control data will be continuous and 
deterministic so I don't believe there will need to be any changes to 
gstreamer's internals except the addition of a new meta data type.

> > Granted the interpolation is a really nice feature but surely it would
> > be better implemented as a set of filter plugins which take a set of
> > parameters and create an analogue data control signal.
> 
> I can see how that would work, but I don't see it as better - just
> different.

See above.

> > > a dparams stream will generally be very low bandwidth and data is
> > >generated in sporadic bursts. The GStreamer infrastructure is really
> > >optimized for streams that produce a mostly-constant stream of data.
> > 
> > In batch situations this is true but not in real-time usage where
> > although changes in the control input are sporadic, the application
> > can have no way of telling when changes are going to occur so the data
> > has to be sampled continuously.
> 
> For true real-time I was imagining that a special RT interpolator would be
> attached to the dynparam instance.  This interpolator would have a fixed
> control rate which matches the buffer boundry.
> 
> The choice and usage of the interpolators is where the real flexibility of
> this approach has advantages IMO.
> 

In RT, how will can there be any interpolation (other than constant) given 
that there's no way of knowing the outcome of the next parameter.

> The minimum of information required would be that you have a linear value
> with lower bound x and upper bound y.  Basically the more information you
> give to the app, the more chance the app has to make sense of the data for
> the user.

Why do they even need bounding values? Surely you should just be able to 
feed whatever values you like to the elements and leave it up to the them 
to behave sensibly given out-of-range data.

> > Even if I could say it was all of these things,
> > what about when new dimensions are introduced. 
> 
> Units of measurement and unit converters should be pluggable - like 
elements
> and types (and interpolators).  It would probably be correct if the core
> gstreamer had minimal builtin units and other units + converters could be
> added by plugins.

I still don't understand the purpose of giving control data dimensions. If 
nothing else it just makes things much more confusing for the user.

> > I see the need for applying parametized envelopes but I believe that
> > is at most 1/4 of the control data picture. 
> 
> > Generating control data
> > from oscillators
> 
> Again, you could write an interpolator to do LFO - it would simple take
> waveform, frequency, amplitude and offset values and away you go.  You 
could
> even envelope each of those parameters so that every parameter could 
change
> over time.  And the UI will give instant feedback because the interpolate
> function would draw a nice wavy line on the screen.

To apply an envelope to a parameter surely means connecting two filters in 
a graph which is exactly what gstreamer already does. That's why I don't 
understand why this should require its own mechanism.

> I don't agree, but I think we have a long way to fall before this thread
> turns into a linux-audio-dev-style flamewar.

Yeah, where's dobey when you need him? :)

Cheers,
Owen

---------------------------------------------------------------------------
     Owen  Fraser-Green             "Hard work never killed anyone,
     owen at discobabe.net              but why give it a chance?"
---------------------------------------------------------------------------