Custom "buffer pool" or "allocator" with avdec_mjpeg

[Thomas Oldbury]

Hi all,

I am trying to get a GStreamer application working where the upstream element (an appsink) will be able to pass a custom buffer pool into an avdec_mjpeg decoder.   The reason for doing this is to avoid a memcpy() operation between the avdec_mjpeg decode and the display.  This implementation is an embedded system with limited resources (it is an FPGA SoC, Xilinx Zynq) and so avoiding a memcpy is essential to maintaining performance.

In the ideal world, I would implement a custom allocator that when presented with a request for a new buffer would choose one of the N buffers available in FPGA space. (N ~14 at present, could be extended later.)  Or I could set up a bufferpool with all these buffers available.  The buffer addresses are fixed: they are mmap'd representations of physical addresses, so unfortunately I can't do this any other way (unless I write my own JPEG decoder, which I'm trying to avoid doing.)

In the ideal world, the JPEG decode would directly decompress the pixel data into the passed framebuffer, achieving zero-copy performance.

The problem I'm having is I've tried setting up an ALLOCATION query like so:

    gst_element_foreach_sink_pad(GST_ELEMENT(gAppSink), cb_padForeach, NULL);

then my cb_padForeach implementation is like this:

    gboolean cb_padForeach(GstElement *element, GstPad *pad, gpointer user_data)
    {
        printf("pad %08x %08x %08x\n", element, pad, user_data);

        GstCaps *resultCaps = gst_caps_new_empty_simple("result");
        GstCaps *caps = gst_caps_new_empty_simple("mycaps");
        gboolean need_pool = FALSE;

        GstQuery *query = gst_query_new_allocation(caps, TRUE);
        gst_pad_query(pad, query);

        gst_query_parse_allocation(query, &resultCaps, &need_pool);

        GstBufferPool *pool = gst_buffer_pool_new();
        gst_buffer_pool_config_set_params(GST_STRUCTURE(pool), NULL, 460800, 36, 36);

        gst_query_add_allocation_pool(query, pool, 460800, 36, 36);

        gst_caps_unref(caps);
        gst_caps_unref(resultCaps);

        return TRUE;
    }

It may be obvious,  but I don't *really* know what I'm doing here;  I'm trying to set the bufferpool up for now to have 36 qty of buffers each 460800 bytes long, but avdec_mjpeg is still using its two buffers (I can see if I print the buffer addresses going into appsink.)  I know this block is entered, but it doesn't appear to influence any behaviour in GStreamer.   N.B. I use 36 in this example as I saw some code in the avdec_X plugin that suggested it wanted to see at least 32 buffers in its ALLOCATION query, but I'm unclear as to whether that is encountered here.  I just set it high for now so I could ensure I wasn't trapping on this.  And I'm aware that this is not using my FPGA buffer addresses, but one step at a time!

I'd appreciate pointers (no pun intended) to any examples of setting up custom bufferpools or allocators in the upstream.

This is my pipeline (in gstreamer syntax, the implementation is in C):

    udpsrc port=5555 ! 'application/x-rtp,payload=26' ! rtpjpegdepay ! queue ! jpegparse ! avdec_mjpeg ! videoconvert ! appsink

I know this pipeline works ok as I can replace appsink with a display sink and I see video from my test pattern generator, also running with GStreamer.

[P.S. I had posted this on IRC, but as I was using a web-client I lost any chat history, and I've just moved house so I've been offline for a week, so apologies if I'm repeating myself.]

Thanks in advance for any advice,
Tom


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220905/7360d4c8/attachment.htm>