[Mesa-dev] Sampling DRM_FORMAT_YUYV in GLSL

[Tapani Pälli]

On 05/09/2017 12:14 PM, Volker Vogelhuber wrote:
> Hi,
> 
> first sorry, for missing the subject in my mail to the mailing list, 
> then thanks
> for the hint with the "ext_image_dma_buf_import-sample_yuv". Unfortunately
> things don't become clearer. In the samples as far as I can see, there 
> is also
> only one sampler defined in the shader. So how are the different planes 
> accessed
> in the shader? In the example only a simple copy is done:
> gl_FragColor = texture2D(sampler, texcoords);
> In the comment in intel_screen.c it says:
> 
> /* For YUYV buffers, we set up two overlapping DRI images and treat
>      * them as planar buffers in the compositors.  Plane 0 is GR88 and
>      * samples YU or YV pairs and places Y into the R component, while
>      * plane 1 is ARGB and samples YUYV clusters and places pairs and
>      * places U into the G component and V into A.  This lets the
>      * texture sampler interpolate the Y components correctly when
>      * sampling from plane 0, and interpolate U and V correctly when
>      * sampling from plane 1. */
> 
> So how are the pixels transfered from the YUYV memory to the vec4 in the 
> shader?
> Do I have to calculate the chroma values by interpolating myself based 
> on the
> current texture coordinate? Or is it done automatically in some way? 
>  From my
> experience the texture call only returns the values from plane0 which 
> leads me to the
> suspicion that I have to interpolate myself. But why is there then the 
> plane 1 which
> don't seem to be accessible in the shader?
> 
> BTW: I'm using the OES_EGL_image extension not the 
> OES_EGL_image_external, so my
> sampler is sampler2D not samplerExternalOES but that shouldn't make a 
> difference should it?

IMO that is a big difference as samplerExternalOES does the YUV2RGB 
conversion and returns RGB values for you.

> And I use "texture" instead of "texture2D" because I'm on OpenGL 3.3, 
> but that does not seem
> to cause any differences.
> 
> On 09.05.2017 06:37, Tapani Pälli wrote:
>> Hi;
>>
>> Take a look at Piglit test "ext_image_dma_buf_import-sample_yuv", 
>> (https://cgit.freedesktop.org/piglit). Test creates EGLImage from dma 
>> buf, binds to a texture and then samples this in a shader with 
>> samplerExternalOES type from GL_OES_EGL_image_external extension.
>>
>>
>> On 05/09/2017 01:57 AM, Volker Vogelhuber wrote:
>>> I'm currently trying to render a V4L2 image with OpenGL
>>> on an Intel Apollo Lake using Linux 4.10 and Mesa 13.
>>> Supprisingly I noticed that importing a DMABUF with format
>>> DRM_FORMAT_YUYV into OpenGL using
>>>    eglCreateImage/glEGLImageTargetTexture2DOES
>>> worked out of the box. But I noticed that there seem to be a split into
>>> two textures when importing the buffer. At least the nplanes
>>> variable in the __DRIimage's planar_format is set to 2 and in the
>>> intel_screen.c there is a comment for __DRI_IMAGE_FOURCC_YUYV:
>>> "For YUYV buffers, we set up two overlapping DRI images
>>>    and treat them as planar buffers in the compositors."
>>> So far so good, but how do I access the second texture in the GLSL
>>> shader. I only created one texture object before calling
>>> glEGLImageTargetTexture2DOES with the EGLImage I got from
>>> eglCreateImage. And using the GLSL function texture( source, texCoord 
>>> ) on this
>>> texture samples only the Y and U channel, what seems obvious
>>> as the first plane's texture is created as GL_RG texture.
>>> Can anyone explain to me, how one accesses the second plane.
>>> And if there is an example somewhere how the two planes
>>> have to be sampled to convert the values back to RGB, it would
>>> be nice if someone can send a link to such an example.
>>>
>>> Thanks
>>> _______________________________________________
>>> mesa-dev mailing list
>>> mesa-dev at lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
>>>
>