[Mesa-dev] GBM YUV planar support

[Rob Clark]

On Thu, Jun 2, 2016 at 2:52 PM, Rob Herring <robh at kernel.org> wrote:
> On Thu, Jun 2, 2016 at 1:31 PM, Rob Clark <robdclark at gmail.com> wrote:
>> On Thu, Jun 2, 2016 at 1:51 PM, Rob Herring <robh at kernel.org> wrote:
>>> As discussed on irc yesterday, I've been looking at adding YUV planar
>>> (YV12) to GBM as it is a requirement for Android gralloc. It is
>>> possible to disable the requirement as that is what the Android
>>> emulator and android-x86 do. But that results in un-optimized s/w CSC.
>>>
>>> Given most/all Android targeted h/w can support YUV overlays (except
>>> virgl?), I only need to allocate, map, and import (to GBM only)
>>> buffers. The outputting of YUV would remain the responsibility of HWC
>>> (also a missing feature in drm_hwcomposer), and the gpu never touches
>>> the YUV buffers.
>>>
>>> With that, I see a couple of options:
>>>
>>> For allocation, at some level we need to translate to a single buffer
>>> perhaps using R8 format. This could be done in gralloc, GBM, gallium
>>> ST, or individual drivers. Also, somewhere we'd have to adjust stride
>>> or height. I don't know what assumptions like the U or V stride is
>>> half the Y stride are acceptable? Trying to propagate per plane stride
>>> and offsets all the way down to the drivers looks difficult.
>>>
>>> Then for importing, we can translate the planes to R8/GR88 and use the
>>> import support Stanimir is doing[1]. Again, the question is at what
>>> level to do this: either gralloc or GBM? The complicating factor here
>>> is I don't think we want to end up with 2 GBM BOs. So maybe GBM BOs
>>> need to support multiple DRIImages? However, it seems that we're
>>> creating 2 ways to import planar buffers either as a single DRIimage
>>> with planes (as i965 does) or a DRIimage per plane.
>>
>> hmm, 2 gbm bo's (and 2 DRIImages) seems kind of ideal if you actually
>> did want to use them w/ gl (as two textures, doing CSC in shader)..
>> although I'm not sure to what extent that breaks the android world (if
>> it was exposed in GL as two textures).
>
> But for allocation, you can't just allocate 2 buffers as the planes do
> have defined ordering and offsets. So somewhere you would need to
> create a 2nd BO and attach it to the buffer. Perhaps dupImage() is for
> this purpose?

hmm, does gralloc really care how many gbm buffers s a single gralloc
buffer maps to?

(possibly dumb question.. I don't have the code handy and forgot most
of it as quickly as possible after last time I had to look at it :-P)

>> AFAIU 99% of the time, in practice, android puts YUV on the screen via
>> overlay, but I'd be curious to see, for example, what the shaders used
>> for window transitions look like.  Depending on to what extent it's
>> possible to change android to support the R8+RG88 + frag shader that
>> does CSC, we might end up with no choice but to add direct support for
>> YUV..
>>
>>> Another option is make gralloc open both render and card nodes using
>>> the card GBM device to just allocate dumb buffers for YUV buffers.
>>> This complicates gralloc a bit and the answer is always don't use dumb
>>> buffers. :) However, the assumption here is the buffers are just
>>> scanout buffers.
>>
>> Note fwiw, what we were doing on linux was actually allocating from
>> the v4l viddec device (and importing in to mesa).  In fact I remember
>> there were some problems going in the other direction.. some specific
>> pitch requirements for the UV plane, or something like that.  And also
>> some really strange corruption (not sure if v4l dmabuf export is
>> broken when the device has an iommu??  maybe it was giving the dmabuf
>> importer device page addresses instead of physical addresses??)
>>
>> Possibly that is an argument for actually allocating video decode
>> buffers from the v4l device directly?
>
> Yes, I'd expect that to be the case for h/w decoders. However, I'm
> using s/w decoders ATM.

I wonder if this is a scenario where we should use the vgem drm
driver.. (basically a small stub driver that just allocations buffers
so you can get a dmabuf from sw rendering)

The dma-buf support in vgem was reverted, since at the time we didn't
have any way to sync sw rendering w/ dma-buf, and not sure if anyone
has re-instated it yet.  I suspect vgem hasn't seen much churn (and it
is a pretty small driver), so probably straightforward to just revert
the removal?

BR,
-R

> Rob