How to manage cached dmabuf importer private data?

[Daniel Vetter]

Two possible solutions, depending upon what you actually want to do:

a) You have a static set of buffers shared between a static set of
devices (e.g. a bunch of yuv frames in a picture pipeline), and you
only want to cut down the (re-)import overhead: Just make your
userspace more intelligent and cache the dam-bufs with an established
caching there instead of trying to pull off clever tricks in the
kernel.

b) You actually have a dynamic set of buffers with constantly changing
buffer sharing sturctures and you want to opportunistically hold onto
as many buffers (with their established device attachments) as
possible without incuring premature OOM: This needs an madvise ioctl
on the dma-buf to mark buffers as purgeable. To actually make this
useful you also need eviction support (to be able to remove the cached
attachements under memory pressure before freeing the actual backing
storage). And maybe also weak file descriptor references to allow the
transfer of the backing storage ownership between multiple processes
without ugly races. You'll get to wrestle Al Viro for this ...

I highly recommend to exhaust a), which doesn't need any kernel
changes, before even considering b) ;-)

Cheers, Daniel

On Tue, Feb 25, 2014 at 9:40 AM, Hiroshi Doyu <hdoyu at nvidia.com> wrote:
> Hi,
>
> We have a problem about how to manage cached dmabuf importer private
> data, where to keep, how to reuse and how to clean up.
>
> We want to keep some data in dmabuf importer side until an buffer is
> free'ed actually since a buffer can be reused again later in that
> importer subsystem so that that cache data doesn't have to be
> regenerated. This can be considered as some kind of caching this data.
>
> The scenario is:
>
>  (1)  Exporter passes a dmabuf to Importer.
>  (2)  Importer attaches a dev to a dmabuf.
>  (3)  Importer generates some data for a buffer for its own use.
>  (4)  Importer finishes its use of a buffer.
>  (5)  Importer detaches a dev from a dmabuf.
>  (6)  Again, Exporter passes a dmabuf fd to the same Importer.
>  (7)  Again, Importer attaches a dev to a dmabuf.
>  (8)  Importer wants to use the previously cached data from (2) without regenerating.
>  (9)  Again, Importer detaches a dev from a dmabuf.
>  (10) Exporter free's a buffer along with a cached data from (2)/(8).
>
> At first I considered to use attachmenet private data, but apparently
> a life time of the attachment isn't equal to one of a buffer. A buffer
> lives longer than an attachment. Also Neither private data from dmabuf
> nor from attachment are for /Importer/. They are for Exporter's use
> from the comment in the header file.
>
> /**
>  * struct dma_buf - shared buffer object
>  ....
>  * @priv: exporter specific private data for this buffer object.
>  */
>
> /**
>  * struct dma_buf_attachment - holds device-buffer attachment data
>  ...
>  * @priv: exporter specific attachment data.
>  ...
>  */
>
> This leads to the following 2 questions:
>
> One question is how to clean up the cached data at (10) since there's
> no way for Importer to trigger clean up at that time.  I am
> considering to embed an /notifier/ in dmabuf when it's called at
> dmabuf release. Importer could register any callback in that
> notifier. At least this requires a dmabuf to have an notifier to be
> called at release. Does this sound acceptable? Or can we do the same
> outside of dmabuf framework? If there's more appropriate way, please
> let me know since I'm not so familier with drm side yet.
>
> Another question is where to keep that cached data. Usually that data
> is only valid within Impoter subsystem. So Imoporter could keep the
> list of that data in it as a global list along with a dmabuf
> pointer. When a dmabuf is imported, Importer can look up a global list
> if it's already cached. This list needs to be kept till a buffer is
> free'ed.
>
> Those can be implemented in the dmabuf exporter backend but we want to
> allow multiple allocators/exporters to do the same, and I want to
> avoid having something related to importer in exporter side.
>
> Any comment would be really appreciated.



-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch