[Bug 776927] New: [PATCH] Accept GLMemory buffers

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=776927

            Bug ID: 776927
           Summary: [PATCH] Accept GLMemory buffers
    Classification: Platform
           Product: GStreamer
           Version: git master
                OS: All
            Status: NEW
          Severity: normal
          Priority: Normal
         Component: gstreamer-vaapi
          Assignee: gstreamer-bugs at lists.freedesktop.org
          Reporter: mattfischer84 at gmail.com
        QA Contact: gstreamer-bugs at lists.freedesktop.org
                CC: bsreerenj at gmail.com, vjaquez at igalia.com
     GNOME version: ---

Created attachment 342994
  --> https://bugzilla.gnome.org/attachment.cgi?id=342994&action=edit
Patch to enable GLMemory buffers

Currently the only way to import a GLMemory buffer into a VAAPI plugin is via a
software download.  However there are API's available to wrap a GL texture's
contents into a VASurface without performing a software download, so the
plugins should support it.

This patch provides an implementation of this procedure.  It seems to work in
my use-case, however there are a couple of issues still to resolve:

1. I've had to force the chroma_type to GST_VAAPI_CHROMA_TYPE_YUV420, since the
video format is of type RGBA and that is not supported by the display. 
Curiously, however, the colors in the encoded video come out correctly.  It
looks to me like the chroma type on the surface must be getting used, which
makes me wonder why we need to bother setting the chroma type on the display at
all.  I tried removing the VAConfigAttribRTFormat attribute completely in
config_create() when making the context, and things still seemed to work
properly.  What is the correct way to handle this case?

2. The key mechanism here in transferring a texture into a surface is the
ability to go Texture -> EGLImage -> DRM handle -> Surface.  All of the
components of this path already existed in the codebase, so it was relatively
straightforward to do, however this obviously doesn't work on non-EGL
platforms.  I don't know enough about GLX to know whether there is some
equivalent that could be done there.  For the moment, I've set it up so that if
the texture cannot be converted to a surface, it fails gracefully and allows a
software download to take place.  This works, but it would be better if a
mechanism could be implemented to do this on GLX.  Does anybody know if that
can be done?

3. In order to create a display which is GL-capable, I added use of
gst_gl_ensure_element_data()/gst_gl_handle_set_context(), so that the plugin
works the same way that other GL plugins do.  However, I think this steps a bit
on the code which already existed in gst_vaapi_plugin_base_decide_allocation(),
which tried to pull a context out of the GLTextureUploadMeta.  Could this code
simply be deleted now that the plugin does a full context query?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.