DMA-heap driver hints

[John Stultz]

On Mon, Jan 23, 2023 at 5:55 AM Laurent Pinchart
<laurent.pinchart at ideasonboard.com> wrote:
>
> Hi Christian,
>
> CC'ing James as I think this is related to his work on the unix device
> memory allocator ([1]).
>
> [1] https://lore.kernel.org/dri-devel/8b555674-1c5b-c791-4547-2ea7c16aee6c@nvidia.com/
>
> On Mon, Jan 23, 2023 at 01:37:54PM +0100, Christian König wrote:
> > Hi guys,
> >
> > this is just an RFC! The last time we discussed the DMA-buf coherency
> > problem [1] we concluded that DMA-heap first needs a better way to
> > communicate to userspace which heap to use for a certain device.
> >
> > As far as I know userspace currently just hard codes that information
> > which is certainly not desirable considering that we should have this
> > inside the kernel as well.
> >
> > So what those two patches here do is to first add some
> > dma_heap_create_device_link() and  dma_heap_remove_device_link()
> > function and then demonstrating the functionality with uvcvideo
> > driver.
> >
> > The preferred DMA-heap is represented with a symlink in sysfs between
> > the device and the virtual DMA-heap device node.
>
> I'll start with a few high-level comments/questions:
>
> - Instead of tying drivers to heaps, have you considered a system where
>   a driver would expose constraints, and a heap would then be selected
>   based on those constraints ? A tight coupling between heaps and
>   drivers means downstream patches to drivers in order to use
>   vendor-specific heaps, that sounds painful.

Though, maybe it should be in that case. More motivation to get your
heap (and its users) upstream. :)


>   A constraint-based system would also, I think, be easier to extend
>   with additional constraints in the future.

I think the issue of enumerating and exposing constraints to userland
is just really tough.  While on any one system there is a fixed number
of constraints, it's not clear we could come up with a bounded set for
all systems.
To avoid this back in the ION days I had proposed an idea of userland
having devices share an opaque constraint cookie, which userland could
mask together between devices and then find a heap that matches the
combined cookie, which would avoid exposing specific constraints to
userland, but the processes of using it seemed like such a mess to
explain.

So I think this driver driven links approach is pretty reasonable. I
do worry we might get situations where the drivers ability to use a
heap depends on some other factor (dts iommu setup maybe?), which the
driver might not know on its own, but I think having the driver
special-case that to resolve it would be doable.


> - I assume some drivers will be able to support multiple heaps. How do
>   you envision this being implemented ?

Yeah. I also agree we need to have multiple heap links.

> - Devices could have different constraints based on particular
>   configurations. For instance, a device may require specific memory
>   layout for multi-planar YUV formats only (as in allocating the Y and C
>   planes of NV12 from different memory banks). A dynamic API may thus be
>   needed (but may also be very painful to use from userspace).

Yeah. While I know folks really don't like the static userspace config
model that Android uses, I do fret that once we get past what a
workable heap is, it still won't address what the ideal heap is.

For instance, we might find that the system heap works for a given
pipeline, but because the cpu doesn't use the buffer in one case, the
system-uncached heap is really the best choice for performance. But in
another pipeline with the same devices, if the cpu is reading and
writing the buffer quite a bit, one would want the standard system
heap.

Because userland is the only one who can know the path a buffer will
take, userland is really the best place to choose the ideal allocation
type.

So while I don't object to this link based approach just to allow a
generic userland to find a working buffer type for a given set of
devices, I don't think it will be able to replace having device
specific userland policy (like gralloc), though it's my personal hope
the policy can be formalized to a config file rather then having
device specific binaries.

thanks
-john