[gst-devel] Re: dparams roadmap (AKA dparams are dead, long live control pads)

[Steve Baker]

This looks like a good start. I've got a couple of questions though.

Where will this all live WRT core, plugins and libraries? One scenario I
see is:
- header and base implementation of GstEnvelope lives in core
- The GstElement struct has an envelope pointer (which will be null until
the application assigns an envelope)

Some might complain about the extra bloat this would add to core - if
anyone objects they should do so now...


There should be some other interpolate methods:
- trigger (already mentioned by Stefan, should also handle string values)
- square (to handle logarithmic values)
- inverse square
- sine
These last three are implemented by Parameter Curves in DirectX Media Objects
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dmo/htm/parametercurves.asp

There might be some other ideas in Parameter Curves worth incorporating.
Maybe the best idea is to make the interpolate mode per GstTimedValue
 instead of per GstEnvelopedProperty. This gives a lot more flexibility in
how the curves are defined:
typedef struct _GstTimedValue
{
  GstClockTime time;
  GValue *value;
  GstInterpolateMode interpolation;        // Interpolation mode
} GstTimedValue;

So is it possible to extend the behaviour of GstEnvelope? If so would it
be by subclassing GstEnvelope or by adding hooks to add more interpolation
modes? Either way using an enumeration for the interpolation modes might
not be flexible enough. Also some fancy interpolation modes might need
some extra data per GstTimedValue to fully define behaviour.

An example is an LFO sine "interpolator" where the *value is the
center-value of the sine wave and the frequency and amplitude are defined
seperately and attached somehow to the GstTimedValue. The alternative
would be to punt this complexity to the application, so they have to build
the sine curves out of GstEnvelope primitives.

And some other minor api comments:
- GSList *values; should probably be GList so backwards traversal is easy
- There needs to be functions to remove a GstTimedValue and remove all
GstTimedValues in a given time range
- does gst_envelope_get_all return a copy or the original? is the app
allowed to manipulate the list or is it read-only?


>   So after this weekend's talks and more talks earlier this week with
> thomas and wim, we came up with a new solution that should fit most
> use-cases and be more lightweight and easier to use than the current
> solution.
>
>   Here follows a proposition and a sample header file. All comments
> welcome.
>
>   The idea is to have a GObject (that we'll call GstEnvelope here)
> that allows applications and plugins to easily handle "dynamic
> parameters", or to put it another way "timed values" for any element
> properties.
>
>   This allows with minor modifications to plugins (one function call)
> to allow any plugin to have any of their properties modified through
> time.
>
> ** Application point of view
>
>   The application decides what properties of what element it wishes to
> control (either ahead of time or in real-time).
>   => gst_element_envelope_properties(element, property names...)
>
>   It can also decide how it wants the interpolation of each properties
> of that envelope (default being no interpolation).
>   => gst_envelope_set_interpolation_mode(envelope, property,
> interpolationmode)
>
>   The application can then give ahead of processing some timed values,
> to create a volume envelope for exemple, or give them at any given
> time.
>   => gst_envelope_set(envelope, property, time, value)
>   => gst_envelope_set_many(envelope, property, list of timed values)
>
>   One of the advantage of this solution is that the envelope will
> record all the modifications made to the given properties through
> time, so that if you play again the handled element it will
> automatically use those values (i.e. for ardour-like applications).
>
>   The other advantage is that it avoids the double-recording of values
> and modifications (in the application and in the element), and allows
> the application a range of values through time that are EXACTLY the
> same as the ones being used by the element. This is useful, for
> example, in non-linear editors to graphically represent the property
> changes through time.
>   => gst_envelope_get_value_array[s] ()
>
> ** plugins point of view
>
>   The minimal requirement for plugins is to use a function, just
> before processing data, that sets the properties handled by the
> envelope to their correct values, given a timestamp.
>   => gst_element_sink_values(element, timestamp)
>   => gst_envelope_sink_values(envelope, object, timestamp)
>
>   If the plugin also needs to have property values at a sample level
> (and not buffer) for some properties, it calls a function that will
> fill up an array of values depending on a timestamp, a number of
> samples, and a sample duration.
>   => gst_element_get_value_array[s] ()
>
>   All the functions are of course threadsafe since the object can be
> handled by the application and by the element.
>
>   The header file is attached, If there's anything not clear, please
> do comment on the mailing-list.
>
>     Edward
>