[gst-devel] Re: dparams roadmap (AKA dparams are dead, long live control pads)

[Stefan Kost]

Hi Edward,

another quicky note:

How does the application can find out for which gobject properties it makes 
sense to add an envelop/controller to. If I recall right e.g. sinesource has two 
properties called sync and buffer-size. It wouldn't make sense to expose them to 
  be controlled.
Its bacillay that elemets have static and dynamic properties.

Now I need to go back to bed, as I've catched a bad flu after getting back from 
barcelona :(

Stefan

Edward Hervey wrote:
> Whooops, was too quick in sending the mail, I sent it to steve only :)
> So here is the mail for everybody.
> 
> 
> Hi all,
> 
>   So after this weekend's talks and more talks earlier this week with
> thomas and wim, we came up with a new solution that should fit most
> use-cases and be more lightweight and easier to use than the current
> solution.
> 
>   Here follows a proposition and a sample header file. All comments welcome.
> 
>   The idea is to have a GObject (that we'll call GstEnvelope here)
> that allows applications and plugins to easily handle "dynamic
> parameters", or to put it another way "timed values" for any element
> properties.
> 
>   This allows with minor modifications to plugins (one function call)
> to allow any plugin to have any of their properties modified through
> time.
> 
> ** Application point of view
> 
>   The application decides what properties of what element it wishes to
> control (either ahead of time or in real-time).
>   => gst_element_envelope_properties(element, property names...)
> 
>   It can also decide how it wants the interpolation of each properties
> of that envelope (default being no interpolation).
>   => gst_envelope_set_interpolation_mode(envelope, property, interpolationmode)
> 
>   The application can then give ahead of processing some timed values,
> to create a volume envelope for exemple, or give them at any given
> time.
>   => gst_envelope_set(envelope, property, time, value)
>   => gst_envelope_set_many(envelope, property, list of timed values)
> 
>   One of the advantage of this solution is that the envelope will
> record all the modifications made to the given properties through
> time, so that if you play again the handled element it will
> automatically use those values (i.e. for ardour-like applications).
> 
>   The other advantage is that it avoids the double-recording of values
> and modifications (in the application and in the element), and allows
> the application a range of values through time that are EXACTLY the
> same as the ones being used by the element. This is useful, for
> example, in non-linear editors to graphically represent the property
> changes through time.
>   => gst_envelope_get_value_array[s] ()
> 
> ** plugins point of view
> 
>   The minimal requirement for plugins is to use a function, just
> before processing data, that sets the properties handled by the
> envelope to their correct values, given a timestamp.
>   => gst_element_sink_values(element, timestamp)
>   => gst_envelope_sink_values(envelope, object, timestamp)
> 
>   If the plugin also needs to have property values at a sample level
> (and not buffer) for some properties, it calls a function that will
> fill up an array of values depending on a timestamp, a number of
> samples, and a sample duration.
>   => gst_element_get_value_array[s] ()
> 
>   All the functions are of course threadsafe since the object can be
> handled by the application and by the element.
> 
>   The header file is attached, If there's anything not clear, please
> do comment on the mailing-list.
> 
>     Edward
> 
> 
> ------------------------------------------------------------------------
> 
> /*
>  * gst_envelope.h
>  * 
>  * New dynamic properties
>  */
> 
> /*
>   GstTimedValue
>   
>   a structure for value+time
> */
> 
> typedef struct _GstTimedValue
> {
>   GstClockTime time;
>   GValue *value;
> } GstTimedValue;
> 
> typedef enum
> {
>   GST_INTERPOLATE_NONE;		// steps-like interpolation, default
>   GST_INTERPOLATE_LINEAR;
>   GST_INTERPOLATE_QUADRATIC;
>   GST_INTERPOLATE_CUBIC;
> } GstInterpolateMode;
> 
> /*
>   The instance structure of GstEnvelope
> */
> 
> typedef struct _GstEnvelope
> {
>   GObject parent;
> 
>   GSList *properties;		// List of GstEnvelopedProperty
>   GMutex *lock;
> } GstEnvelope;
> 
> /*
>   GstEnvelopedProperty
> */
> 
> typedef struct _GstEnvelopedProperty
> {
>   gchar *name;			// name of the property
>   GType type;
>   GstInterpolateMode interpolation;	// Interpolation mode
>   GSList *values;		// List of GstTimedValue
> } GstEnvelopedProperty;
> 
> 
> /*
>   gst_envelope_properties
> 
>   Creates a new GstEnvelope for the given object's properties
> */
> 
> GstEnvelope *gst_envelope_properties (GObject * object,
> 				      gchar * first_property_name, ...);
> 
> /*
>   gst_element_envelope_properties
> 
>   Convenience function for GstElement
> 
>   Creates a GstEnvelope that allows you to dynamically control one, or more, GstElement properties.
>   If the given GstElement already has a GstEnvelope, it adds the given properties to the envelope and
>   returns the existing envelope.
> 
>   Returns : The GstEnvelope with which the user can control the given properties dynamically or NULL if
>   one or more of the given properties aren't available in the given element.
> */
> 
> GstEnvelope *gst_element_envelope_properties (GstElement * element,
> 					      gchar * first_property_name,
> 					      ...);
> 
> 
> /*
>   gst_unenvelope_properties
> 
>   Removes the given object properties from the envelope
> 
>   Returns : FALSE if one of the given property isn't handled by the envelope, TRUE otherwise
> */
> 
> gboolean gst_unenvelope_properties (GstEnvelope * envelope,
> 				    GObject * object,
> 				    gchar * first_property_name, ...);
> 
> /*
>   gst_element_unenvelope_properties
> 
>   Convenience function for GstElement
> 
>   Removes the given element's properties from it's envelope
> 
>   Returns : FALSE if one of the given property names isn't handled by the envelope, TRUE otherwise
> */
> 
> gboolean gst_element_unenvelope_properties (GstElement * element,
> 					    gchar * first_property_name, ...);
> 
> /*
>   gst_element_get_envelope
>   element :
> 
>   Returns : the envelope handling some of the given element's properties, NULL if no envelope
> */
> 
> GstEnvelope *gst_element_get_envelope (GstElement * element);
> 
> 
> /*
>   gst_element_set_envelope
> 
>   Sets the envelope on the given GstElement
> 
>   Returns : FALSE if the GstElement already has an envelope, TRUE otherwise
> */
> 
> gboolean gst_element_set_envelope (GstElement * element,
> 				   GstEnvelope * envelope);
> 
> /*
>   gst_envelope_set*
> 
>   Functions used to set the values of given envelope-handled properties
> 
>   Returns FALSE if the values couldn't be set (ex : properties not handled by envelope), TRUE otherwise
> */
> 
> gboolean gst_envelope_set (GstEnvelope * envelope, gchar * property_name,
> 			   GstClockTime time, GValue * value);
> 
> gboolean gst_envelope_set_many (GstEnvelope * envelope, gchar * property_name,
> 				GSList * timedvalues);
> 
> 
> /*
>   gst_envelope_get
> 
>   Returns : the GValue of the property at the given time, or NULL if the property isn't handled by the envelope
> */
> 
> GValue	*gst_envelope_get (GstEnvelope * envelope, gchar * property_name,
> 			   GstClockTime time);
> 
> /*
>   gst_envelope_get_all
> 
>   Returns : The list of GstTimedValue for the given property, or NULL if the property isn't handled by the envelope
> */
> 
> const GSList	*gst_envelope_get_all (GstEnvelope * envelope, gchar * property_name);
> 
> 
> /*
>   gst_envelope_sink_values
> 
>   Sets the properties of the element, according to the envelope that (maybe) handles them 
>   and for the given timestamp.
> 
>   Returns : TRUE if the envelope values could be applied to the object properties, FALSE otherwise
> */
> 
> gboolean gst_envelope_sink_values (GstEnvelope * envelope, GObject * object,
> 				   GstClockTime timestamp);
> 
> 
> /*
>   gst_element_sink_values
> 
>   convenience macro for GstElement
> 
>   Returns : same thing as gst_envelope_sink_values
> */
> 
> #define gst_element_sink_values(element, timestamp) \
>         ((element)->envelope) ? \
> 	gst_envelope_sink_values ((element)->envelope, G_OBJECT(element), timestamp) :\
>         TRUE;
> 
> /*
>   gst_*_get_value_array[s]
> 
>   Functions to be able to get an array of values for one or more given element properties
> 
>   If the GstValueArray->values array is NULL, it will be created by the function.
>   The type of the values in the array are the same as the property's type.
> 
>   the gst_element_* functions are just convenience functions for GstElement
> 
>   Returns : TRUE if the given array(s) could be filled, FALSE otherwise
> */
> 
> typedef struct _GstValueArray
> {
>   gchar *property_name;
>   gint nbsamples;		// Number of samples requested
>   gint64 sample_interval;	// Interval between each sample
>   gpointer *values;		// pointer to the array (so it can be filled if NULL)
> } GstValueArray;
> 
> gboolean gst_envelope_get_value_arrays (GstEnvelope * envelope,
> 					gint64 timestamp,
> 					GSList * value_arrays);
> 
> gboolean gst_envelope_get_value_array (GstEnvelope * envelope,
> 				       gint64 timestamp,
> 				       GstValueArray * value_array);
> 
> 
> gboolean gst_element_get_value_arrays (GstElement * element,
> 				       gint64 timestamp,
> 				       GSList * value_arrays);
> 
> gboolean gst_element_get_value_array (GstElement * element,
> 				      gint64 timestamp,
> 				      GstValueArray * value_array);
> 
> 
> /*
>   gst_envelope_set_interpolation_mode
>   envelope :
>   property_name :
>   mode : interpolation mode
> 
>   Sets the given interpolation mode on the given property.
> 
>   Returns : TRUE if the property is handled by the envelope, FALSE otherwise
> */
> 
> gboolean gst_envelope_set_interpolation_mode (GstEnvelope * envelope,
> 					      gchar * property_name,
> 					      GstInterpolateMode mode);
>