[Mesa-dev] GBM YUV planar support

[Kristian Høgsberg]

On Thu, Jun 2, 2016 at 11:52 AM, Rob Herring <robh at kernel.org> wrote:
> On Thu, Jun 2, 2016 at 1:31 PM, Rob Clark <robdclark at gmail.com> wrote:
>> On Thu, Jun 2, 2016 at 1:51 PM, Rob Herring <robh at kernel.org> wrote:
>>> As discussed on irc yesterday, I've been looking at adding YUV planar
>>> (YV12) to GBM as it is a requirement for Android gralloc. It is
>>> possible to disable the requirement as that is what the Android
>>> emulator and android-x86 do. But that results in un-optimized s/w CSC.
>>>
>>> Given most/all Android targeted h/w can support YUV overlays (except
>>> virgl?), I only need to allocate, map, and import (to GBM only)
>>> buffers. The outputting of YUV would remain the responsibility of HWC
>>> (also a missing feature in drm_hwcomposer), and the gpu never touches
>>> the YUV buffers.
>>>
>>> With that, I see a couple of options:
>>>
>>> For allocation, at some level we need to translate to a single buffer
>>> perhaps using R8 format. This could be done in gralloc, GBM, gallium
>>> ST, or individual drivers. Also, somewhere we'd have to adjust stride
>>> or height. I don't know what assumptions like the U or V stride is
>>> half the Y stride are acceptable? Trying to propagate per plane stride
>>> and offsets all the way down to the drivers looks difficult.
>>>
>>> Then for importing, we can translate the planes to R8/GR88 and use the
>>> import support Stanimir is doing[1]. Again, the question is at what
>>> level to do this: either gralloc or GBM? The complicating factor here
>>> is I don't think we want to end up with 2 GBM BOs. So maybe GBM BOs
>>> need to support multiple DRIImages? However, it seems that we're
>>> creating 2 ways to import planar buffers either as a single DRIimage
>>> with planes (as i965 does) or a DRIimage per plane.
>>
>> hmm, 2 gbm bo's (and 2 DRIImages) seems kind of ideal if you actually
>> did want to use them w/ gl (as two textures, doing CSC in shader)..
>> although I'm not sure to what extent that breaks the android world (if
>> it was exposed in GL as two textures).
>
> But for allocation, you can't just allocate 2 buffers as the planes do
> have defined ordering and offsets. So somewhere you would need to
> create a 2nd BO and attach it to the buffer. Perhaps dupImage() is for
> this purpose?

We added dupImage for the cases where you import some resource
(wl_buffer or EGLImage) that is also backed by a __DRIimage to a
gbm_bo. We wanted the lilfetime of the gbm_bo to be independent of the
imported resource lifetime.

>> AFAIU 99% of the time, in practice, android puts YUV on the screen via
>> overlay, but I'd be curious to see, for example, what the shaders used
>> for window transitions look like.  Depending on to what extent it's
>> possible to change android to support the R8+RG88 + frag shader that
>> does CSC, we might end up with no choice but to add direct support for
>> YUV..
>>
>>> Another option is make gralloc open both render and card nodes using
>>> the card GBM device to just allocate dumb buffers for YUV buffers.
>>> This complicates gralloc a bit and the answer is always don't use dumb
>>> buffers. :) However, the assumption here is the buffers are just
>>> scanout buffers.
>>
>> Note fwiw, what we were doing on linux was actually allocating from
>> the v4l viddec device (and importing in to mesa).  In fact I remember
>> there were some problems going in the other direction.. some specific
>> pitch requirements for the UV plane, or something like that.  And also
>> some really strange corruption (not sure if v4l dmabuf export is
>> broken when the device has an iommu??  maybe it was giving the dmabuf
>> importer device page addresses instead of physical addresses??)
>>
>> Possibly that is an argument for actually allocating video decode
>> buffers from the v4l device directly?
>
> Yes, I'd expect that to be the case for h/w decoders. However, I'm
> using s/w decoders ATM.

I agree with Rob for HW decoders, that's how libva also works for
Intel. It seems like the problem left to solve is how to allocate a
gbm_bo that can be mapped and written to by a SW decoder and then
(potentially exported to another process) and used as a texture by GL.
My feeling is that we just want to teach gbm_bo_create() about the YUV
formats and then allow gbm_bo_map to map separate planes (maybe
through new flags). Whether or not to use multiple BOs or not can be a
decision the driver makes then. As long as the planes have proper
alignments, I don't see a problem just using one BO and creating
multiple textures from that.

Kristian