Try to address the DMA-buf coherency problem

[Christian König]

Am 30.11.22 um 11:30 schrieb Daniel Vetter:
> On Fri, Nov 25, 2022 at 11:40:04AM -0500, Nicolas Dufresne wrote:
>> Le mercredi 23 novembre 2022 à 17:33 +0100, Daniel Vetter a écrit :
>>> On Wed, Nov 23, 2022 at 10:33:38AM +0200, Pekka Paalanen wrote:
>>>> On Tue, 22 Nov 2022 18:33:59 +0100
>>>> Christian König <christian.koenig at amd.com> wrote:
>>>>
>>>>> We should have come up with dma-heaps earlier and make it clear that
>>>>> exporting a DMA-buf from a device gives you something device specific
>>>>> which might or might not work with others.
>>>>>
>>>>> Apart from that I agree, DMA-buf should be capable of handling this.
>>>>> Question left is what documentation is missing to make it clear how
>>>>> things are supposed to work?
>>>> Perhaps somewhat related from Daniel Stone that seems to have been
>>>> forgotten:
>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fdri-devel%2F20210905122742.86029-1-daniels%40collabora.com%2F&data=05%7C01%7Cchristian.koenig%40amd.com%7C45786a08e6dc4af2816508dad2bdf957%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638054011293521624%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GjsoJGNoPozTS2SWeeirURzQatI5vfl9%2B%2BfzoavgTbw%3D&reserved=0
>>>>
>>>> It aimed mostly at userspace, but sounds to me like the coherency stuff
>>>> could use a section of its own there?
>>> Hm yeah it would be great to land that and then eventually extend. Daniel?
>> There is a lot of things documented in this document that have been said to be
>> completely wrong user-space behaviour in this thread. But it seems to pre-date
>> the DMA Heaps. The document also assume that DMA Heaps completely solves the CMA
>> vs system memory issue. But it also underline a very important aspect, that
>> userland is not aware which one to use. What this document suggest though seems
>> more realist then what has been said here.
>>
>> Its overall a great document, it unfortunate that it only makes it into the DRM
>> mailing list.
> The doc is more about document the current status quo/best practices,
> which is very much not using dma-heaps.
>
> The issue there is that currently userspace has no idea which dma-heap to
> use for shared buffers, plus not all allocators are exposed through heaps
> to begin with. We had this noted as a todo item (add some device->heap
> sysfs links was the idea), until that's done all you can do is hardcode
> the right heaps for the right usage in userspace, which is what android
> does. Plus android doesn't have dgpu, so doesn't need the missing ttm
> heap.

Hui? Why do you think we should have a TTM heap in the first place?

As far as I can see we need three different ways of allocation:
1. Normal system memory.
2. CMA based.
3. Device specific.

When any of the participating devices needs CMA then user space must use 
the CMA heap, when any of the participating devices needs device 
specific memory then user space must use device specific memory (from 
that device).

It still can be that two devices can't talk with each other because both 
needs the buffer to be allocated from device specific memory for a 
particular use case, but at least all the cases for both none CPU cache 
coherent ARM devices as well as device specific memory for scanout 
should be handled with this.

Regards,
Christian.

>
> But yeah the long-term aspiration also hasn't changed, because the
> dma-heap todo list is also very, very old by now :-/
> -Daniel