[Bug 743345] glimagesink: Add support for dmabuf

Thu Jan 29 06:30:20 PST 2015

https://bugzilla.gnome.org/show_bug.cgi?id=743345
  GStreamer | gst-plugins-bad | unspecified

--- Comment #4 from Nicolas Dufresne (stormer) <nicolas.dufresne at collabora.co.uk> 2015-01-29 14:30:15 UTC ---
(In reply to comment #3)
> I can see you have not added code here:
> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-libs/gst/gl/egl/gsteglimagememory.c,
> any reason ?  (Also as a side note I can see about eglimage some duplicated
> code added recently in gstglupload since last time I looked)

I have proposed to make the uploader create EGLImage with GstEglImageMemory 
with this code as an intermediate step and then use the uploader there. That
would mean adding some logic so we know what texture target is supported. This
would get rid of the duplicate.

> 
> Anyway, with omx/eglimage_with_context/gltexture, the sink can suggest a pool
> of eglimage_with_gl_context_and_texture that are later filled by the upstream
> decoder. But I do not see either a pool of eglimage in your case. It seems you
> create an eglimage for every buffer. I do not see the egldestroy btw.

The decoder in question here does not know about EGL. It's simply a DMABUF
producer. I don't think we should add a dependency to EGL in there (also you'll
get plugins-good linking to plugins-bad issue). I think the EGL pool should
remain internal to the sink.

Creating new image for every buffer is unfortunate, but as attaching the DMABUF
isn't separate from the creation of the EGLImage, I don't think it's possible
to optimized this easily (not with an upstream pool). I'll leave the egldestroy
part to lubosz for comments.

> 
> In your scenario I do not see any pool of eglimage_no_context_and_with_dmaf_buf
> .
> Also here I do not see any mention to "dma" here
> http://cgit.collabora.com/git/user/lubosz/gst-plugins-bad.git/tree/ext/gl/gstglimagesink.c?id=cc89ca45b172919ccdd9c68e237e6be3abe3ec06#n1206
> .

This is mainly my requirement. Because of the nature of DMABUF, even if you
take all the possible care to negotiate thing, you still have no guaranty that
you will be able to import them.

While I think it will be needed eventually to have a dmabuf caps feature, I
think most our implementation should not rely on this. glimagesink has shaders
doing color convertionm hence is a good candidate to not depend on that.

The general idea is that it will decide the best upload path when it receives
the buffer. If dmabuf does not work, then it will fallback at uploading and
color converting the usual way in a transparent manner (we can cache the
results for performance reason obviously).

> 
> Depending on the allocator selected by the upstream decoder, the downstream
> pool will be setup with or without eglimage, see
> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-libs/gst/gl/gstglbufferpool.c#n149
>  and  then
> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-libs/gst/gl/gstglbufferpool.c#n245
> through gst_egl_image_memory_setup_buffer.
> 
> Well here to distinguish either eglimage_with_context_and_texture or
> eglimage_no_context_and_with_dmaf_buf  we could make it a bit generic by using
> the existing GstAllocationParams/GstMemoryFlags with some improvements or add
> gstgl API used by the decoder (see
> http://cgit.freedesktop.org/gstreamer/gst-omx/tree/omx/gstomxvideodec.c#n41)
> That would avoid to introduce memory:dmabuf and re-use memory:EGLImage

I don't think that fits. In this design, upstream decoder (omx) know about EGL
Images. In our use case, upstream only produce sysmem backed by an FD which is
a DMABUF. DMABUF is the common denominator here.

> 
> Finally about the multiple plane case, even if the driver supports it, maybe
> you need to create one eglimage per plane as done here:
> https://github.com/raspberrypi/userland/blob/master/host_applications/linux/apps/raspicam/RaspiTex.h#L162
> (or here
> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-libs/gst/gl/egl/gsteglimagememory.c#n480
> but not yet tested) . In your code I can see only one call to eglcreateimage,
> but again I may have missed something.

This has been discussed as an intermediate. The design of DMABUF import is that
you can have up to 3 planes attached to a single EGLImage. In this case, the
texture target is TEXTURE_OES, and color conversion is taken care by the
driver. Daniel explained all the cases. The intermediate (when multiple planes
does not work) would be to bind each dmabuf plane to an EGLImage. The unknown
so far is if the set of DRM format will be sufficient to represent 8 bits/1
component and 16bits / 2 components formats, or at least if there is a way
around so the sampler can behave in a way that works with our shaders. This
will need research, and is mostly an optimization for driver that only do
single plane per EGLImage.

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.