metadata propagation in gsttransform plugins

Thu Jan 19 15:08:51 UTC 2017

Hi Dimitrios,

On Thu, Jan 19, 2017 at 12:47:43PM +0100, Dimitrios Katsaros wrote:
> If you have no tags in the gst_meta_api_type_register then the metadata
> should pass all the element checks. So adding the video tag and removing
> memory would only change something if you were already defining you own
> tags, like e.g. GPS.

Well, I have now removed our own tag and the newly added "video" tag,
calling thus gst_meta_api_type_register with an empty tag array,
and now our meta_transform function (always copy) is called, and our
metadata are propagated auto-magically to the encoded buffers :)

> 
> looking at this function:
> https://github.com/Freescale/gstreamer-imx/blob/master/src/vpu/encoder_base.c#L504
> frames passed to finish frame are not the same as the input_frame param of
> the function. Your metadata is probably on the input frame and not being
> handled since the call to the meta transform happens in
> gst_video_encoder_finish_frame. I think you will have to implement the
> metadata handling for the finished frames of that element.

It seems thus that I won't need to implement metadata handling in
encoder_base.c

Thank you for your help

Philippe
> 
> On Thu, Jan 19, 2017 at 10:29 AM, Philippe De Muyter <phdm at macq.eu> wrote:
> 
> > On Thu, Jan 19, 2017 at 09:41:35AM +0100, Dimitrios Katsaros wrote:
> > > And i forgot to answer your specific questions, sorry about that :p
> > >
> > > 1) There is a default propagation mechanism implemented in every plugin.
> > It
> > > usually goes by the name "transform_meta". You should not have to
> > implement
> > > it assuming that elements inherit from the base classes. What you do need
> > > to implement is a transformation function within your metadata.
> > >
> > > 2) metadata lifetime is linked with the buffer it is associated with.
> > When
> > > tee propagates a buffer it increases the buffers reference count for
> > every
> > > branch. So you should not have to worry about the reference counting for
> > > the metadata.
> >
> > After the 'tee' I have only one instance of our metadata with a refcount
> > set
> > to 2.  After the videoencoder, in the specific case where the metadata are
> > invariant per frame, is that instance merely attached to the encoded
> > buffer or
> > must it be copied by the transformation function of our metadata ?
> >
> > Philippe
> >
> > >
> > > DImitrios
> > >
> > > On Thu, Jan 19, 2017 at 9:33 AM, Dimitrios Katsaros <
> > patcherwork at gmail.com>
> > > wrote:
> > >
> > > > I am not able to find the code implementing that specific encoder on
> > the
> > > > fly so I will try and work with whtever assumptions I can make about
> > > > encoders in general and your post.
> > > >
> > > > You will need to make sure that your metadata implements a
> > transformation
> > > > function, the encoder implements a transformation function and that the
> > > > tags of the metadata are correctly set for the specific transformation.
> > > > Assuming that the imxvpuenc_mjpeg inherits the videoencoder class and
> > does
> > > > not override the transform_meta function, you will need to make sure
> > that
> > > > the meta tags include "video" and do *not* include "memory"
> > > >
> > > >
> > > > Dimitrios
> > > >
> > > > On Thu, Jan 19, 2017 at 12:47 AM, Philippe De Muyter <phdm at macq.eu>
> > wrote:
> > > >
> > > >> Hi all,
> > > >>
> > > >> I have read that since the 1.6 version of gstreamer frame tags are
> > > >> preserved
> > > >> by each plugin.  My questions below are targeted at the propagation by
> > > >> GstVideoEncoders of memory- and frame-size- independent tags, such as
> > gps
> > > >> data,
> > > >> exposure, gain that could vary from frame to frame, and so on.
> > > >>
> > > >> I have the following pipeline :
> > > >>
> > > >> gst-launch-1.0 imxv4l2videosrc ! tee name=t ! queue ! \
> > > >>         imxvpuenc_mjpeg ! mprintmeta logname=1 ! fakesink i\
> > > >>         t. ! queue ! imxvpuenc_h264 ! mprintmeta logname=2 ! fakesink
> > > >>
> > > >> where imxv4l2videosrc has been modified to add custom metadata, and
> > > >> mprintmeta is debugging to show those metadata.  I notice that the
> > > >> metadata are not present after the encoders
> > > >>
> > > >> - Is the propagation mechanism implemented outside of the plugins, or
> > must
> > > >> it be implemented in each plugin ? In that case, is a generic
> > propagation
> > > >> code
> > > >> available, that we could add e.g. to gstreamer-imx (imx hardware
> > > >> accelerated)
> > > >> encoder plugins ?
> > > >>
> > > >> - Is GstMetaFreeFunction only called after all instances are unused,
> > or is
> > > >> there a refcount accessible inside the definition of
> > GstMetaFreeFunction
> > > >> for a new metadata type, to avoid that one branch after a 'tee'
> > destroys a
> > > >> metadata that's still used by the other branch ?
> > > >>
> > > >> Thanks in advance
> > > >>
> > > >> Philippe
> > > >>
> > > >> --
> > > >> Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140
> > > >> Bruxelles
> > > >> _______________________________________________
> > > >> gstreamer-devel mailing list
> > > >> gstreamer-devel at lists.freedesktop.org
> > > >> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > >>
> > > >
> > > >
> >
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > gstreamer-devel at lists.freedesktop.org
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
> >
> > --
> > Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140
> > Bruxelles
> > _______________________________________________
> > gstreamer-devel mailing list
> > gstreamer-devel at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >

> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

-- 
Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles