[Mesa-dev] Sampling DRM_FORMAT_YUYV in GLSL

[Philipp Zabel]

On Tue, 2017-05-09 at 12:31 +0300, Tapani Pälli wrote:
> 
> On 05/09/2017 12:29 PM, Tapani Pälli wrote:
> > 
> > 
> > On 05/09/2017 12:14 PM, Volker Vogelhuber wrote:
> >> Hi,
> >>
> >> first sorry, for missing the subject in my mail to the mailing list, 
> >> then thanks
> >> for the hint with the "ext_image_dma_buf_import-sample_yuv". 
> >> Unfortunately
> >> things don't become clearer. In the samples as far as I can see, there 
> >> is also
> >> only one sampler defined in the shader. So how are the different 
> >> planes accessed
> >> in the shader? In the example only a simple copy is done:
> >> gl_FragColor = texture2D(sampler, texcoords);
> >> In the comment in intel_screen.c it says:
> >>
> >> /* For YUYV buffers, we set up two overlapping DRI images and treat
> >>      * them as planar buffers in the compositors.  Plane 0 is GR88 and
> >>      * samples YU or YV pairs and places Y into the R component, while
> >>      * plane 1 is ARGB and samples YUYV clusters and places pairs and
> >>      * places U into the G component and V into A.  This lets the
> >>      * texture sampler interpolate the Y components correctly when
> >>      * sampling from plane 0, and interpolate U and V correctly when
> >>      * sampling from plane 1. */
> >>
> >> So how are the pixels transfered from the YUYV memory to the vec4 in 
> >> the shader?
> >> Do I have to calculate the chroma values by interpolating myself based 
> >> on the
> >> current texture coordinate? Or is it done automatically in some way? 
> >>  From my
> >> experience the texture call only returns the values from plane0 which 
> >> leads me to the
> >> suspicion that I have to interpolate myself. But why is there then the 
> >> plane 1 which
> >> don't seem to be accessible in the shader?
> >>
> >> BTW: I'm using the OES_EGL_image extension not the 
> >> OES_EGL_image_external, so my
> >> sampler is sampler2D not samplerExternalOES but that shouldn't make a 
> >> difference should it?
> > 
> > IMO that is a big difference as samplerExternalOES does the YUV2RGB 
> > conversion and returns RGB values for you.
> 
> I have to add that "this is how I think it works", I haven't tried this 
> myself :)

This is correct. The OES_EGL_image_external extension states:

   "Sampling an external texture will return an RGBA vector in the same
    colorspace as the source image.  If the source image is stored in YUV
    (or some other basis) then the YUV values will be transformed to RGB
    values. (But these RGB values will be in the same colorspace as the
    original image."

Whereas when using a GL_TEXTURE_2D target, you have to do the conversion
yourself, sampling from to separate textures specified from two
EGLImages (one RG88 for Y, and one ARGB for UV, as described above).

regards
Philipp