Custom "buffer pool" or "allocator" with avdec_mjpeg

[Nicolas Dufresne]

Hi,

Le lun. 5 sept. 2022, 14 h 03, Thomas Oldbury via gstreamer-devel <
gstreamer-devel at lists.freedesktop.org> a écrit :

> Hi all,
>
> I am trying to get a GStreamer application working where the upstream
> element (an appsink) will be able to pass a custom buffer pool into an
> avdec_mjpeg decoder.   The reason for doing this is to avoid a memcpy()
> operation between the avdec_mjpeg decode and the display.  This
> implementation is an embedded system with limited resources (it is an FPGA
> SoC, Xilinx Zynq) and so avoiding a memcpy is essential to maintaining
> performance.
>
> In the ideal world, I would implement a custom allocator that when
> presented with a request for a new buffer would choose one of the N buffers
> available in FPGA space. (N ~14 at present, could be extended later.)  Or I
> could set up a bufferpool with all these buffers available.  The buffer
> addresses are fixed: they are mmap'd representations of physical addresses,
> so unfortunately I can't do this any other way (unless I write my own JPEG
> decoder, which I'm trying to avoid doing.)
>
> In the ideal world, the JPEG decode would directly decompress the pixel
> data into the passed framebuffer, achieving zero-copy performance.
>
> The problem I'm having is I've tried setting up an ALLOCATION query like
> so:
>
>     gst_element_foreach_sink_pad(GST_ELEMENT(gAppSink), cb_padForeach,
> NULL);
>

As you suspected, this is not going to do what you want. Appsink have a
single static sink pas called "sink", just get it by name and add a pad
probe callback instead. Check the query type and handle the allocation
query.

Regards,
Nicolas



> then my cb_padForeach implementation is like this:
>
>     gboolean cb_padForeach(GstElement *element, GstPad *pad, gpointer
> user_data)
>     {
>         printf("pad %08x %08x %08x\n", element, pad, user_data);
>
>         GstCaps *resultCaps = gst_caps_new_empty_simple("result");
>         GstCaps *caps = gst_caps_new_empty_simple("mycaps");
>         gboolean need_pool = FALSE;
>
>         GstQuery *query = gst_query_new_allocation(caps, TRUE);
>         gst_pad_query(pad, query);
>
>         gst_query_parse_allocation(query, &resultCaps, &need_pool);
>
>         GstBufferPool *pool = gst_buffer_pool_new();
>         gst_buffer_pool_config_set_params(GST_STRUCTURE(pool), NULL,
> 460800, 36, 36);
>
>         gst_query_add_allocation_pool(query, pool, 460800, 36, 36);
>
>         gst_caps_unref(caps);
>         gst_caps_unref(resultCaps);
>
>         return TRUE;
>     }
>
> It may be obvious,  but I don't *really* know what I'm doing here;  I'm
> trying to set the bufferpool up for now to have 36 qty of buffers each
> 460800 bytes long, but avdec_mjpeg is still using its two buffers (I can
> see if I print the buffer addresses going into appsink.)  I know this block
> is entered, but it doesn't appear to influence any behaviour in GStreamer.
>  N.B. I use 36 in this example as I saw some code in the avdec_X plugin
> that suggested it wanted to see at least 32 buffers in its ALLOCATION
> query, but I'm unclear as to whether that is encountered here.  I just set
> it high for now so I could ensure I wasn't trapping on this.  And I'm aware
> that this is not using my FPGA buffer addresses, but one step at a time!
>
> I'd appreciate pointers (no pun intended) to any examples of setting up
> custom bufferpools or allocators in the upstream.
>
> This is my pipeline (in gstreamer syntax, the implementation is in C):
>
>     udpsrc port=5555 ! 'application/x-rtp,payload=26' ! rtpjpegdepay !
> queue ! jpegparse ! avdec_mjpeg ! videoconvert ! appsink
>
> I know this pipeline works ok as I can replace appsink with a display sink
> and I see video from my test pattern generator, also running with
> GStreamer.
>
> [P.S. I had posted this on IRC, but as I was using a web-client I lost any
> chat history, and I've just moved house so I've been offline for a week, so
> apologies if I'm repeating myself.]
>
> Thanks in advance for any advice,
> Tom
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220911/8558149b/attachment.htm>