plugin writing: base class for format conversion

[Guennadi Liakhovetski]

Hi Michael,

On Mon, 19 Jul 2021, Michael Gruner wrote:

> Glad it worked!
>
> If you want to overlay a picture (like the thermal bar in that GIF), you
> can check the gdkpixbufoverlay
> <https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-good/html/gst-plugins-good-plugins-gdkpixbufoverlay.html>
> element. If you also want to add a border to the image so that the
> overlay doesn’t cover part of the image, you may look at the videobox
> <https://gstreamer.freedesktop.org/documentation/videobox/index.html?gi-language=c>
> element. By setting the “right” property to a negative value, you’ll add
> a black border to the right of the image.

Thanks, I'll check those! Meanwhile I realised, that I have one more
difficulty with this: the image I'm getting from the camera is 32x24
pixels, so, if I just extend it similar to that GIF I still only have 24
pixels vertically which isn't a lot for temperature text :-) I usually add
a videoscale element to the pipeline after my conversion plugin, that
works well, but that doesn't help with that bar - I want to add it after
scaling, and the actual processing must happen before scaling, it must
access original camera data.

One approach I could take is write a second plugin to apply after scaling,
but apart from the image that plugin would need some additional data from
the first one - at least the temperature range. My current plan is to pass
those two numbers in metadata lines below the image and hope that those
numbers aren't modified by the videoscale plugin...

Are there better ways to achieve this? Can I "call" videoscale internally
in my conversion plugin, then I wouldn't need two plugins. Or can I pass
metadata in some way more robust than image pixels?

Thanks
Guennadi

> > On 18 Jul 2021, at 05:09, Guennadi Liakhovetski <g.liakhovetski at gmx.de> wrote:
> >
> > On Sat, 17 Jul 2021, Michael Gruner wrote:
> >
> >> The default is to only allow the same format. You need to reimplement
> >> the transform_caps virtual method in order to change the format. (note
> >> that transform_caps is a basetransform method).
> >
> > Great, this works! Thanks very much! I've got a colour-coded image now.
> > Ideally - if not too much work - I'd also like to get numeric temperature
> > information like on the side bar in the GIF on
> > https://github.com/makerportal/raspi-thermal-cam or maybe in an additional
> > in-frame metadata channel. I found a discussion here
> > https://developer.ridgerun.com/wiki/index.php/GStreamer_and_in-band_metadata
> > but that doesn't seem to have made it to the mainline. I guess the most
> > user-friendly would be adding a side bar like in the former link. It
> > should be possible to extend the width of the image and overlay that kind
> > of information, right?
> >
> > Thanks
> > Guennadi
> >
> >>> On 17 Jul 2021, at 13:50, Guennadi Liakhovetski <g.liakhovetski at gmx.de> wrote:
> >>>
> >>> Hi Michael,
> >>>
> >>>> On Sat, 17 Jul 2021, Michael Gruner wrote:
> >>>>
> >>>> Hi Guennadi
> >>>>
> >>>> The videoconvert element can already handle this conversion for you:
> >>>>
> >>>> gst-launch-1.0 v4l2src ! videoconvert ! video/x-raw,format=RGB ! …
> >>>
> >>> Yes, I know about videoconvert, but I need to do some special processing
> >>> of thermal data. For the algorithms to work I have to access the raw data,
> >>> which is output by the v4l2 driver in GRAY16_BE format (1-to-1 from the
> >>> camera), and I want to transform that to a colour gradient similar to well
> >>> known thermal images, so my plugin cannot just stay within GRAY16_BE and
> >>> use videoconvert to convert that to RGB - you'd just get a gray image of
> >>> course.
> >>>
> >>>> If you, for other reason, need to write your own element, you probably
> >>>> want to use the videofilter base class. It’s a specialization of the
> >>>> basetransform class for video. The important virtual method for you to
> >>>> implement is “transform_caps”, that will allow you to specify the input
> >>>> and output caps respectively.
> >>>
> >>> The videofilter class is "allowed" to convert between formats? I thought
> >>> it would be, so I actually tried it. But I was unable to get it to input
> >>> and output different formats. I was only able to get it to work with the
> >>> same format. Are you sure that is supposed to work?
> >>>
> >>> Thanks
> >>> Guennadi
> >>>
> >>>> Michael
> >>>>
> >>>>>> On 17 Jul 2021, at 05:05, Guennadi Liakhovetski via gstreamer-devel <gstreamer-devel at lists.freedesktop.org> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I'm trying to write a plugin to convert data from a thermal camera. It
> >>>>> should take data from the v4l2src source in GRAY16_BE format and convert
> >>>>> it to an RGB video output. I'm looking at gst-element-maker but I'm not
> >>>>> finding any suitable base class that would take video/x-raw as input and
> >>>>> generate the same as output and would convert between format. Do I have to
> >>>>> write a plugin from scratch? Or should I use basetransform? Any tutorials
> >>>>> for that?
> >>>>>
> >>>>> Thanks
> >>>>> Guennadi
> >>>>> _______________________________________________
> >>>>> gstreamer-devel mailing list
> >>>>> gstreamer-devel at lists.freedesktop.org
> >>>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >>>>
> >>
>
>