[gst-devel] Understanding and Reading GObject code Re: VideoFilter(transformation element) implementation

Thu Apr 8 09:43:25 CEST 2010

On Thu, 2010-04-08 at 00:46 +0300, alex wrote:
> On 07/04/2010 19:44, Edward Hervey wrote:
> >> So why is it still in there?  Gstreamer is hard enough to figure out as it is.
> >
> >    Define 'there'. I can't see any mention of that obsolete video filter
> > template in git of gstreamer core, -base or -template.
> >
> >     Edward
> >
> > P.S. Asking for it to be removed instead of complaining as you're doing
> > might have, you know... a better chance of it being fixed.
> http://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/gst/videofilter/gstvideotemplate.c

  Indeed, that template shouldn't be used. In fact, as we'll see below,
implemeting a video filter doesn't really require a template, there
isn't much code involved once you put aside all the GObject template
required.

  The tip from Stefan is still valid. Have a look at videobalance in
that directory for example, it's a *rather* clean example for an
in-place filter.

  How to read such code ? Here's the step-by-step of reading and
understanding GObject-based code. We'll be going over it in roughly the
same order as the code excution.

  (Anybody having issues/troubles *reading* and understanding
GStreamer/GObject code should read this too and then head to a more
extensive GObject documentation to understand the finer details [1])

* GType

  The trick to understand ANY GObject classes implementation (and what
is needed/used) is to realize that everything GObject related starts
with a GType.

  A GType is *the* identifier/alias for a class. C doesn't have the
notion of classes, so GObject offers a system to be able to
differentiate classes (and have inheritance, etc... like any other
object-oriented system). If inheritance and classes are unknown to you,
go have a quick read on wikipedia about object-oriented programming.

  For every class, we need a way to be able to get its GType. The
GObject way of doing that is to define a macro to get to it:
gstvideobalance.h:
#define GST_TYPE_VIDEO_BALANCE (gst_video_balance_get_type())

==> THIS is the entry point to reading any gobject-code. Anything
related to a GObject class starts here !. Look at the rest of the macros
in that header, they all use GST_TYPE_VIDEO_BALANCE.
    If you look at other header files in GStreamer core (like
gstelement.h, gstbin.h, ....) they all have those macros pointing to a
_get_type() function.

 In the GStreamer plugin context, when you a plugin is read/loaded (see
plugin_init at the bottom of gstvideobalance.c) you also just pass along
that GType to GStreamer ("hey GStreamer, here's the GType of a
GstElement class you should offer to users").

** GObject type registration

  Let's carry on reading the code, and have a look at _get_type.

* gst_video_balance_get_type
  This method return the GType correponding to this class. If not
present (i.e. the very first time it's called), it will register it.
i.e. define the basics of this type.

  The g_type_register_*() line gives the overview:
   * We give it the parent type. It's parent GType here is
GST_TYPE_VIDEO_FILTER (it could have been GST_TYPE_BASE_TRANSFORM for
the sake of clarity, but no biggie). The most basic class will derive
from G_TYPE_OBJECT.
   * It's well know name is "GstVideoBalance". This is mainly used for
debugging.
   * Some information about our GType is registered (video_balance_info)
     * methods to be called the very first time a GType is used (i.e.
the first time an instance of this class (or a subclass) will be needed.
     * details about the difference between base_init and class_init can
be seen in GObject tutorials, suffice to say they're both called before
the very first time an instance of this type will be created. Not all
classes use the _base_init, but all classes that need to override some
class vmethods need to have a _class_init and do it there.
     * we also give the method which will be called whenever an instance
of this type will be created (gst_video_balance_init).

  You can ignore the parts about interfaces for the time being since
they're not essential for this, but if you understood what
g_type_register does, it's basically the same idea but for interfaces.

** GObject class initialization

  This is where we will refine the behaviour of our class, i.e. override
class-global behaviour (methods, class global properties, ...)

* gst_video_balance_class_init
  => it overrides the set_property/get_property methods of GObjectClass,
those are the methods that will be called when you want to read/set one
of the properties you exposed.
  => it installs some properties that can be used from the outside
(Those are the ones that appear in gst-inspect, that you can set on
gst-launch or with g_object_set())
  => It implements a GObjectClass finalize method (this is where you
should clean up everything you allocated in your instance)
  => It overrides the GstTransformClass 'set_caps' method, this is
because it wants to know the configured width and height of the video
buffers we will receive.
  => Finally, and most important in our case, it implements the
'transform_ip' method of GstBaseTransform. We are hereby saying that we
can modify the buffers in-place (we don't need to create a new one).
Useful if your input and output buffer are the same size.

** GObject instance initialization

  This method (defined in the type registration, here
gst_video_balance_init) will be called whenever a new instance of our
class is created.
  If instance-specific data needs to be allocated/initialized, it should
be done here.
  Don't forget to free any allocated data in the 'finalize' method of
your class.

** Carrying on with our transform element

  Everything from now on is specific to the videobalance class, based on
all the initialization we did in the few steps above.

  We implemented a GstBaseTransform 'set_caps' method
  gst_video_balance_set_caps:
  * We remember in our instance what the configured width/height are and
return TRUE (provided those properties are present in the caps)

  Our class doesn't modify the caps and implements a 'transform_ip'
method, so that method will be called for every buffer coming in.

  gst_video_balance_transform_ip:
  * ignore the passthrough related parts for the time being, and the
checks for sizes.
  What does it do ? Takes the buffer data... and calls the actual
transformation method (i.e. what's specific to your element) with the
buffer data and the configured width/height.

*** What now ?

  This showed the breakdown of how/what code gets called in a GObject
class. Hopefully this should help people be able to *read* gstreamer
plugins a bit quicker, and help read/understand the GStreamer reference
manual.

  If you want an example of a video filter that *does* create new
buffers, read in the same way the code to gstvideoflip in the same
directory. It's a 700 line self-contained file.

  Hope this helps,

    Edward

[1] : http://library.gnome.org/devel/gobject/stable/

> 
> I'll be very happy if it will be fixed. For now I found ~3 working 
> solutions (really 3 variations of 1 solution). As for now i derive my 
> plugin from GstElement, and variants are:
> 1) create temp buffer once in _start() method and free it in _finalize()
> 2a) create temp buffer, use it and free it in _transform() with GLib 
> memory management functions
> 2b) create output buffer (gst_buffer_copy), make transformations on it 
> and free input buffer (gst_buffer_unref) in _transform() with those 
> gst-specific functions.
> 
> I think 1-st method better, because buffer allocated and freed only once.
> 
> But i still think, that it is not the fully right way.
> 
> P.S. If in near future i will gain better understanding of this i'll try 
> to fix template by myself.
> 
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel