Fwd: [gst-devel] Re: dparams roadmap (AKA dparams are dead, long live control pads)

[Edward Hervey]

Hi,

On Thu, 24 Feb 2005 14:16:23 +1300 (NZDT), Steve Baker
<steve at stevebaker.org> wrote:
> This looks like a good start. I've got a couple of questions though.
>
> Where will this all live WRT core, plugins and libraries? One scenario I
> see is:
> - header and base implementation of GstEnvelope lives in core
> - The GstElement struct has an envelope pointer (which will be null until
> the application assigns an envelope)

  I agree this is the best way to do it. We'll be making a sample
implementation with stefan before to see if it suits everybody.

>
> Some might complain about the extra bloat this would add to core - if
> anyone objects they should do so now...
>
> There should be some other interpolate methods:
> - trigger (already mentioned by Stefan, should also handle string values)
> - square (to handle logarithmic values)
> - inverse square
> - sine
> These last three are implemented by Parameter Curves in DirectX Media Objects
> http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dmo/htm/parametercurves.asp

  Agreed those are interesting interpolation methods, but we first
need to get the basic interpolations working (Trigger, linear, square
(what I called quadratic), cubic). All those interpolations only need
TimedValues (no extra parameters per point).

>
> There might be some other ideas in Parameter Curves worth incorporating.
> Maybe the best idea is to make the interpolate mode per GstTimedValue
>  instead of per GstEnvelopedProperty. This gives a lot more flexibility in
> how the curves are defined:
> typedef struct _GstTimedValue
> {
>   GstClockTime time;
>   GValue *value;
>   GstInterpolateMode interpolation;        // Interpolation mode
> } GstTimedValue;

  This seems to be an overkill way of doing things. Most people will
be using the same interpolation method all the time.
  Also changing interpolation method for a property would mean having
to change the interpolation variable for all timedvalues.
  And how would you decide how to do the interpolation between two
points that have different interpolation methods. For example: Point A
(linear interpolation) and point B (cubic interpolation) ?

>
> So is it possible to extend the behaviour of GstEnvelope? If so would it
> be by subclassing GstEnvelope or by adding hooks to add more interpolation
> modes? Either way using an enumeration for the interpolation modes might
> not be flexible enough. Also some fancy interpolation modes might need
> some extra data per GstTimedValue to fully define behaviour.
>
> An example is an LFO sine "interpolator" where the *value is the
> center-value of the sine wave and the frequency and amplitude are defined
> seperately and attached somehow to the GstTimedValue. The alternative
> would be to punt this complexity to the application, so they have to build
> the sine curves out of GstEnvelope primitives.

  One way I see of doing this would be to:
  * make GstEnvelopedProperty handle the list of TimedValues,
  * have the interpolation methods mentionned above as structures of
function pointers (hooks) that are automatically selected when
changing the interpolation method. The content of that structure is
public.
  * Add an extra function for selecting interpolation where the user
can specify a function pointers structure of his choice (and that
automatically sets the interpolation to GST_INTERPOLATE_USER).
  * Allow the user to create "derivates" of the GstTimedValue (à la
GObject). Ex : typedef struct {GstTimedValue value; gboolean
extra_flag} GstTimedValueMyInterpolation;

>
> And some other minor api comments:
> - GSList *values; should probably be GList so backwards traversal is easy

  Agreed.

> - There needs to be functions to remove a GstTimedValue and remove all
> GstTimedValues in a given time range

  Agreed.

> - does gst_envelope_get_all return a copy or the original? is the app
> allowed to manipulate the list or is it read-only?

  Good point. For the time being I made it return the original (const
GSList *) , but I guess it would be safer to make it return a copy.

>
>
> >   So after this weekend's talks and more talks earlier this week with
> > thomas and wim, we came up with a new solution that should fit most
> > use-cases and be more lightweight and easier to use than the current
> > solution.
> >
> >   Here follows a proposition and a sample header file. All comments
> > welcome.
> >
> >   The idea is to have a GObject (that we'll call GstEnvelope here)
> > that allows applications and plugins to easily handle "dynamic
> > parameters", or to put it another way "timed values" for any element
> > properties.
> >
> >   This allows with minor modifications to plugins (one function call)
> > to allow any plugin to have any of their properties modified through
> > time.
> >
> > ** Application point of view
> >
> >   The application decides what properties of what element it wishes to
> > control (either ahead of time or in real-time).
> >   => gst_element_envelope_properties(element, property names...)
> >
> >   It can also decide how it wants the interpolation of each properties
> > of that envelope (default being no interpolation).
> >   => gst_envelope_set_interpolation_mode(envelope, property,
> > interpolationmode)
> >
> >   The application can then give ahead of processing some timed values,
> > to create a volume envelope for exemple, or give them at any given
> > time.
> >   => gst_envelope_set(envelope, property, time, value)
> >   => gst_envelope_set_many(envelope, property, list of timed values)
> >
> >   One of the advantage of this solution is that the envelope will
> > record all the modifications made to the given properties through
> > time, so that if you play again the handled element it will
> > automatically use those values (i.e. for ardour-like applications).
> >
> >   The other advantage is that it avoids the double-recording of values
> > and modifications (in the application and in the element), and allows
> > the application a range of values through time that are EXACTLY the
> > same as the ones being used by the element. This is useful, for
> > example, in non-linear editors to graphically represent the property
> > changes through time.
> >   => gst_envelope_get_value_array[s] ()
> >
> > ** plugins point of view
> >
> >   The minimal requirement for plugins is to use a function, just
> > before processing data, that sets the properties handled by the
> > envelope to their correct values, given a timestamp.
> >   => gst_element_sink_values(element, timestamp)
> >   => gst_envelope_sink_values(envelope, object, timestamp)
> >
> >   If the plugin also needs to have property values at a sample level
> > (and not buffer) for some properties, it calls a function that will
> > fill up an array of values depending on a timestamp, a number of
> > samples, and a sample duration.
> >   => gst_element_get_value_array[s] ()
> >
> >   All the functions are of course threadsafe since the object can be
> > handled by the application and by the element.
> >
> >   The header file is attached, If there's anything not clear, please
> > do comment on the mailing-list.
> >
> >     Edward
> >
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>

More comments to follow, don't hesitate to comment...

   Edward