Try to address the DMA-buf coherency problem

[Daniel Vetter]

On Fri, Dec 02, 2022 at 04:27:08PM +0100, Christian König wrote:
> Am 30.11.22 um 11:30 schrieb Daniel Vetter:
> > On Fri, Nov 25, 2022 at 11:40:04AM -0500, Nicolas Dufresne wrote:
> > > Le mercredi 23 novembre 2022 à 17:33 +0100, Daniel Vetter a écrit :
> > > > On Wed, Nov 23, 2022 at 10:33:38AM +0200, Pekka Paalanen wrote:
> > > > > On Tue, 22 Nov 2022 18:33:59 +0100
> > > > > Christian König <christian.koenig at amd.com> wrote:
> > > > > 
> > > > > > We should have come up with dma-heaps earlier and make it clear that
> > > > > > exporting a DMA-buf from a device gives you something device specific
> > > > > > which might or might not work with others.
> > > > > > 
> > > > > > Apart from that I agree, DMA-buf should be capable of handling this.
> > > > > > Question left is what documentation is missing to make it clear how
> > > > > > things are supposed to work?
> > > > > Perhaps somewhat related from Daniel Stone that seems to have been
> > > > > forgotten:
> > > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fdri-devel%2F20210905122742.86029-1-daniels%40collabora.com%2F&data=05%7C01%7Cchristian.koenig%40amd.com%7C45786a08e6dc4af2816508dad2bdf957%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638054011293521624%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GjsoJGNoPozTS2SWeeirURzQatI5vfl9%2B%2BfzoavgTbw%3D&reserved=0
> > > > > 
> > > > > It aimed mostly at userspace, but sounds to me like the coherency stuff
> > > > > could use a section of its own there?
> > > > Hm yeah it would be great to land that and then eventually extend. Daniel?
> > > There is a lot of things documented in this document that have been said to be
> > > completely wrong user-space behaviour in this thread. But it seems to pre-date
> > > the DMA Heaps. The document also assume that DMA Heaps completely solves the CMA
> > > vs system memory issue. But it also underline a very important aspect, that
> > > userland is not aware which one to use. What this document suggest though seems
> > > more realist then what has been said here.
> > > 
> > > Its overall a great document, it unfortunate that it only makes it into the DRM
> > > mailing list.
> > The doc is more about document the current status quo/best practices,
> > which is very much not using dma-heaps.
> > 
> > The issue there is that currently userspace has no idea which dma-heap to
> > use for shared buffers, plus not all allocators are exposed through heaps
> > to begin with. We had this noted as a todo item (add some device->heap
> > sysfs links was the idea), until that's done all you can do is hardcode
> > the right heaps for the right usage in userspace, which is what android
> > does. Plus android doesn't have dgpu, so doesn't need the missing ttm
> > heap.
> 
> Hui? Why do you think we should have a TTM heap in the first place?

[sorry for the really late reply]

Extremely late reply, but p2p buffer sharing when you want it in vram is
only something ttm can provide you. And if the goal is to have all buffer
sharing needs served by dma-heaps, then eventually that'll be needed too.

But probably totally fine to just not have it for another few years
(decades?).

> As far as I can see we need three different ways of allocation:
> 1. Normal system memory.
> 2. CMA based.
> 3. Device specific.
> 
> When any of the participating devices needs CMA then user space must use the
> CMA heap, when any of the participating devices needs device specific memory
> then user space must use device specific memory (from that device).
> 
> It still can be that two devices can't talk with each other because both
> needs the buffer to be allocated from device specific memory for a
> particular use case, but at least all the cases for both none CPU cache
> coherent ARM devices as well as device specific memory for scanout should be
> handled with this.

Yeah on ARM this should be enough.
-Daniel

> 
> Regards,
> Christian.
> 
> > 
> > But yeah the long-term aspiration also hasn't changed, because the
> > dma-heap todo list is also very, very old by now :-/
> > -Daniel
> 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch