GStreamer Compute

[Nirbheek Chauhan]

On Mon, Apr 8, 2019 at 7:40 AM Aaron Boxer <boxerab at gmail.com> wrote:
> Is there any existing element I could use as a template?
> One twist is that GPUs give the highest performance in asynchronous mode, so
> the element will need to pull buffers from upstream, schedule them to
> the GPU, and only push the processed buffers downstream when the GPU
> completes.  Also, it would be good to have the option of keeping the memory
> on the device, in order to apply a series of kernels without costly move
> to host and back to device.
>

The gstgl library does exactly all this. See, for example, inside
gst-plugins-base/gst-libs/gst/gl/gstglwindow.c,
gst_gl_window_default_send_message() which for instance runs all GL
functions inside the glib main context attached to the window. You can
probably do something similar.

For keeping memory on the device, that's already supported by
GstBuffer and by GstMemory, for instance GL and DMA-BUF memories are
always on the device. You can add a new caps type for device-side
compute buffers. Matthew (ystreet) might've spent some time on this
already.

Cheers,
Nirbheek