[Bug 776927] vaapi: plugins: accept GLMemory buffers from upstream

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=776927

--- Comment #14 from Víctor Manuel Jáquez Leal <vjaquez at igalia.com> ---
(In reply to Matt Fischer from comment #13)
> (In reply to Matthew Waters (ystreet00) from comment #12)
> > That would probably be best performed by a buffer pool somewhere.
> 
> Yeah, I can add a buffer pool, but we need more than just that here.  You
> don't just want to preserve the buffers you made, you also need to be able
> to look up the output buffer associated with a specific input buffer when it
> comes in.  That way, when buffer A comes in, you can pull out the output
> buffer associated with its texture, and when buffer B comes in, you can get
> its buffer, etc.  I don't know that buffer pools provide a way to do this,
> do they?
> 
> I think you'd need some kind of map where you could save them all, but then
> you get into tricky issues of when to clean them up and stuff like that. 
> Maybe it'd be enough to just drop them all when the element stops or
> something.

Perhaps you don't need a buffer pool, but "associates" the input buffer with a
created output buffer, using gst_mini_object_set_qdata(). If the input buffer
has already an associated output buffer, just reuse it. In theory, it should
have the same dmabuf fd if the texture hasn't changed. This is what GstGLUpload
does.

> 
> > I'm not sure either however it is certainly possible for dmabuf's to be
> > unmappable by software elements which is where the caps feature is useful in
> > order to explicitly request (or not) a certain feature.
> 
> My understanding is that when the dmabuf is not mappable, you are supposed
> to use memory:DMABuf, to tell the downstream element that it must use
> special API's to access the data.  But if the dmabuf can be mapped, then you
> aren't supposed to use any special features, since it can be trivially
> treated like a software buffer if the downstream element doesn't know any
> better.  Is that correct?
> 
> I don't see any documentation in MESA_image_dma_buf_export about whether the
> dmabufs it produces are mappable, though, so I'm not sure what to say for
> this case.

An approach would be to try to map the first generated dmabuf buffer. If maps
fails, memory:DMABuf capfeature is negotiated, otherwise no features.

> 
> > An option is to push a completely new memory type from gldownload so that
> > symem mapping will always work (as far as is permitted by GL and dmabuf).
> 
> That would probably work, although you would need to make sure that
> gst_is_dmabuf_memory() and gst_dmabuf_memory_get_fd() still work properly on
> it, so that the existing VAAPI code can still import the dmabuf directly.
> 
> > Another completely different option is for vaapi to propose a glbufferpool
> > that already deals with creating the egl images attached to textures.
> > (probably not)
> 
> You would probably need to invent a GstGLTextureDownloadMeta that the
> bufferpool could put on the buffers, so that gldownload knows how to tell it
> to perform the download, right?  Might be nice for symmetry with
> GstGLTextureUploadMeta, but sounds like a lot of work.

I don't see this. Though I was one who asked for the merge of
GstGLTextureUploadMeta, now I think that it is not the best idea, it is very
problematic (gl context sharing and so -- not only GstGL elements, but clutter
and maybe others).

> > The other option is for vaapi to do the transformation to eglimages itself
> > although that's less exciting. (probably not)
> 
> That's the way my original patch on this bug did it.  I just added
> memory:GLMemory to all of the elements' input caps, and added code to detect
> getting a GstGLMemory and perform an EGLImage -> DRM buffer -> VASurface
> conversion.
> 
> Now that you can see these two methods side-by-side, do you guys have any
> thoughts on which one would be the better way to go about this?

As you may see, I'm a bit reluctant to clutter gstreamer-vaapi more as it is
now.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.