[Bug 727886] New: GstVideoGLTextureUploadMeta assumes no color conversion is done during upload

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=727886
  GStreamer | gst-plugins-base | git

           Summary: GstVideoGLTextureUploadMeta assumes no color
                    conversion is done during upload
    Classification: Platform
           Product: GStreamer
           Version: git
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: gst-plugins-base
        AssignedTo: gstreamer-bugs at lists.freedesktop.org
        ReportedBy: kiagiadakis.george at gmail.com
         QAContact: gstreamer-bugs at lists.freedesktop.org
     GNOME version: ---


GstVideoGLTextureUploadMeta has a design issue regarding what happens to the
buffer after it is uploaded to the texture(s).

On some hardware, using some hardware-specific method to upload the buffer to
the GPU leaves the data in its original format. It looks like
GstVideoGLTextureUploadMeta has been designed with that case in mind, so for
example eglglessink when it sees that the negotiated format is I420, it expects
the UploadMeta to upload 3 textures (one for each plane) and uses a shader to
convert the colorspace.

However, one some other hardware (like for example imx6), the hardware-specific
method for uploading a buffer from the VPU directly to the GPU also internally
does colorspace conversion and gives only 1 texture as an output, even if the
buffer is YUV. In that case, the sink needs to treat the result as RGB and use
a dummy shader with 1 texture.

Unfortunately, there is no way to negotiate this properly. We could set the
format in the caps to be RGB, but that is wrong because if you map the buffer
before uploading it, the data is not necessarily RGB. Another way would be a
flag in the UploadMeta, but 1) there is no padding to add that in the structure
and 2) the sink should ideally know this information early (during caps
negotiation probably) in order to initialize its internal state before the
first buffer arrives. Maybe the best would be a field in the caps... something
like "uploaded-format=(string)xRGB" ?

Suggestions? Comments?

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.