<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto">Hey Guennadi<div> </div><div>GStreamer has the GstMeta API, which allows you to attach metadata to the buffers which, from your last email, seems to be what you need. If you are using GStreamer 1.20 or later you can use gst_buffer_add_custom_meta. On previous versions, you’ll need to implement your own meta. If you need an example, here’s an implementation that adds video-related metadata to the buffers:</div><div> </div><div><a href="https://github.com/GStreamer/gst-plugins-base/blob/master/gst-libs/gst/video/gstvideometa.c">https://github.com/GStreamer/gst-plugins-base/blob/master/gst-libs/gst/video/gstvideometa.c</a></div><div> <div dir="ltr">The meta approach is what I would personally go for. For completeness of the answer, there is a way to have elements within elements, it’s called a bin. Your element would inherit from GstBin and basically build a sub-pipeline.</div><div id="AppleMailSignature" dir="ltr"> </div><div id="AppleMailSignature" dir="ltr">One final idea is to consider using a third party scaler in your element, opencv or libyuv, for example. Your transform_caps method would change, not only the format, but the size as well. Then on the transform_frame you’d call the scaler after your processing. however, i’d remain principled and avoid mixing the two responsibilities and go for the meta approach. </div><div id="AppleMailSignature" dir="ltr"> </div><div dir="ltr">Michael</div><div dir="ltr"> <blockquote type="cite">On 31 Jul 2021, at 02:42, Guennadi Liakhovetski <g.liakhovetski@gmx.de> wrote: </blockquote></div><blockquote type="cite"><div dir="ltr">Hi Michael, On Mon, 19 Jul 2021, Michael Gruner wrote: <blockquote type="cite">Glad it worked! </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">If you want to overlay a picture (like the thermal bar in that GIF), you </blockquote><blockquote type="cite">can check the gdkpixbufoverlay </blockquote><blockquote type="cite"><https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-good/html/gst-plugins-good-plugins-gdkpixbufoverlay.html> </blockquote><blockquote type="cite">element. If you also want to add a border to the image so that the </blockquote><blockquote type="cite">overlay doesn’t cover part of the image, you may look at the videobox </blockquote><blockquote type="cite"><https://gstreamer.freedesktop.org/documentation/videobox/index.html?gi-language=c> </blockquote><blockquote type="cite">element. By setting the “right” property to a negative value, you’ll add </blockquote><blockquote type="cite">a black border to the right of the image. </blockquote> Thanks, I'll check those! Meanwhile I realised, that I have one more difficulty with this: the image I'm getting from the camera is 32x24 pixels, so, if I just extend it similar to that GIF I still only have 24 pixels vertically which isn't a lot for temperature text :-) I usually add a videoscale element to the pipeline after my conversion plugin, that works well, but that doesn't help with that bar - I want to add it after scaling, and the actual processing must happen before scaling, it must access original camera data. One approach I could take is write a second plugin to apply after scaling, but apart from the image that plugin would need some additional data from the first one - at least the temperature range. My current plan is to pass those two numbers in metadata lines below the image and hope that those numbers aren't modified by the videoscale plugin... Are there better ways to achieve this? Can I "call" videoscale internally in my conversion plugin, then I wouldn't need two plugins. Or can I pass metadata in some way more robust than image pixels? Thanks Guennadi <blockquote type="cite"><blockquote type="cite">On 18 Jul 2021, at 05:09, Guennadi Liakhovetski <g.liakhovetski@gmx.de> wrote: </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">On Sat, 17 Jul 2021, Michael Gruner wrote: </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">The default is to only allow the same format. You need to reimplement </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">the transform_caps virtual method in order to change the format. (note </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">that transform_caps is a basetransform method). </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Great, this works! Thanks very much! I've got a colour-coded image now. </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Ideally - if not too much work - I'd also like to get numeric temperature </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">information like on the side bar in the GIF on </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">https://github.com/makerportal/raspi-thermal-cam or maybe in an additional </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">in-frame metadata channel. I found a discussion here </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">https://developer.ridgerun.com/wiki/index.php/GStreamer_and_in-band_metadata </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">but that doesn't seem to have made it to the mainline. I guess the most </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">user-friendly would be adding a side bar like in the former link. It </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">should be possible to extend the width of the image and overlay that kind </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">of information, right? </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Thanks </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Guennadi </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">On 17 Jul 2021, at 13:50, Guennadi Liakhovetski <g.liakhovetski@gmx.de> wrote: </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Hi Michael, </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">On Sat, 17 Jul 2021, Michael Gruner wrote: </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Hi Guennadi </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">The videoconvert element can already handle this conversion for you: </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">gst-launch-1.0 v4l2src ! videoconvert ! video/x-raw,format=RGB ! … </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Yes, I know about videoconvert, but I need to do some special processing </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">of thermal data. For the algorithms to work I have to access the raw data, </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">which is output by the v4l2 driver in GRAY16_BE format (1-to-1 from the </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">camera), and I want to transform that to a colour gradient similar to well </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">known thermal images, so my plugin cannot just stay within GRAY16_BE and </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">use videoconvert to convert that to RGB - you'd just get a gray image of </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">course. </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">If you, for other reason, need to write your own element, you probably </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">want to use the videofilter base class. It’s a specialization of the </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">basetransform class for video. The important virtual method for you to </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">implement is “transform_caps”, that will allow you to specify the input </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">and output caps respectively. </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">The videofilter class is "allowed" to convert between formats? I thought </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">it would be, so I actually tried it. But I was unable to get it to input </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">and output different formats. I was only able to get it to work with the </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">same format. Are you sure that is supposed to work? </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Thanks </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Guennadi </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Michael </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">On 17 Jul 2021, at 05:05, Guennadi Liakhovetski via gstreamer-devel <gstreamer-devel@lists.freedesktop.org> wrote: </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Hi, </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">I'm trying to write a plugin to convert data from a thermal camera. It </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">should take data from the v4l2src source in GRAY16_BE format and convert </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">it to an RGB video output. I'm looking at gst-element-maker but I'm not </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">finding any suitable base class that would take video/x-raw as input and </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">generate the same as output and would convert between format. Do I have to </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">write a plugin from scratch? Or should I use basetransform? Any tutorials </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">for that? </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Thanks </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Guennadi </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">_______________________________________________ </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">gstreamer-devel mailing list </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">gstreamer-devel@lists.freedesktop.org </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel </blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"> </blockquote><blockquote type="cite"> </blockquote></div></blockquote></div></body></html>