Introduce a new helper framework for buffer synchronization

[Inki Dae]

Hi all,

This post introduces a new helper framework based on dma fence. And the
purpose of this post is to collect other opinions and advices before RFC
posting.

First of all, this helper framework, called fence helper, is in progress
yet so might not have enough comments in codes and also might need to be
more cleaned up. Moreover, we might be missing some parts of the dma fence.
However, I'd like to say that all things mentioned below has been tested
with Linux platform and worked well.

The below link is fence helper codes,

https://git.kernel.org/cgit/linux/kernel/git/daeinki/drm-exynos.git/commit/?h=dma-fence-work&id=59096850b48ba0c5cd944dfc1dc4c2b8a92c12f5

And fence helper is based on the below patches,

http://lists.freedesktop.org/archives/dri-devel/2013-January/033430.html

Fence helper includes easy-to-use user and kernel interfaces for applying a
buffer synchronization mechanism into device driver and user space.

And the below link is example codes for device drivers,

https://git.kernel.org/cgit/linux/kernel/git/daeinki/drm-exynos.git/commit/?h=dma-fence-work&id=750a6bee163882e9f0cda48ce21eca7d18614e1c

And also the below link is user interface relevant codes for user process,

https://git.kernel.org/cgit/linux/kernel/git/daeinki/drm-exynos.git/commit/?h=dma-fence-work&id=d2e30e8d8564b42c838108c86dae23138ddae7be

The dma fence framework[1] had already been posted by Maarten and Rob. And
the dma fence is used with reservation framework.

Simple mechanism of the dma fence is as the followings,
1. one dmabuf has one reservation object when one buffer is exported into a
dmabuf.

2. a thread creates its own fence object and sets the fence object to a
reservation object of a dmabuf: a dmabuf is a buffer a thread wants
accessing.

3. dma fence provides fence_default_wait() to block a thread and
fence_signal() to wake up a thread: a thread is blocked once trying to
access a shared buffer if another fence object had already been set to a
reservation object of a shared buffer. And then a blocked thread is waked
up by a owner when the owner- the owner is a thread accessing the shared
buffer - signals the shared buffer.

PS. Know that all buffers to be shared should be exported into dmabuf fd.

However, dma fence just provides basic interfaces for buffer
synchronization between DMA and DMA. Therefore developers should consider
how interfaces of the dma fence should be called and also the resources of
the dma fence should be managed. Thus, it might not be easy to apply the
dma fence framework into their device drivers.

Moreover, we need user interfaces for buffer synchronization between CPU
and DMA. For example, we cannot aware of when and how CPU tries to access a
shared buffer. So user applications need to notify the fact to dma fence
framework so that the dma fence framework can synchronize the shared buffer
between CPU and DMA.

Fence helper includes the following features and interfaces.

Features.
1. provide optimized buffer synchronization method - we need to check if a
thread tried to access buffer with READ or WRITE, and if a thread tries to
access same buffer with READ or WRITE. The reason is that we can avoid
unnecessary read-lock by buffer synchronization and perform proper cache
operation.

2. provide buffer synchronization to READ, WRITE and DMA access - Fence
helper has three access types: READ, WRITE, and DMA. And these types can be
compounded like below, READ, WRITE, READ | DMA or WRITE | DMA.

3. provide user interfaces for buffer synchronization between CPU and DMA -
Fence helper provides user interfaces and the user interfaces has been
implemented in dma-buf framework. User application can call these
interfaces before and after cpu access.

4. provide cache control method - Fence helper performs proper cache
operation looking before and after of CPU and DMA access. Actual cache
operation is done just before committing a fence after memory access by CPU
or DMA was completed. For cache operation, dma-buf interfaces,
dma_buf_begin_cpu_access() and dma_buf_end_cpu_access(), are used.

5. provide easy-to-use user and kernel interfaces - Fence helper has four
exported interfaces: these can be called by device driver and user
application like below,

Interfaces.
fence_helper_init() - this function creates a new fence helper object
internally and returns the fence helper object.

fence_helper_init_reserve() - this function sets up a shard buffer into the
fence helper object. And mutiple shared buffers can be set to there.

fence_helper_commit_reserve() - this function synchronizes a shared buffer
checking fence helper objects already committeds and commits a fence helper
object to a shared buffer. The checking means that a caller waits for all
threads, which committed their fence helper objects, signal the shared
buffer.  And the commit means that the caller is using a shared buffer so
the shared buffer access by another thread isn't permitted: any thread
trying to access the shared buffer will be blocked until a owner signals
the shared buffer.

fence_helper_signal() - this function signals a shared buffer. This means
that the shared buffer access by a caller has been completed so other
threads can access the shared buffer. at this moment, if there are blocked
threads, they are waked up and then they commit their fences to the same
shared buffer: they was blocked at fence_helper_commit_reserve() so are
waked up at there.

And tutorial.
        when setting up dma relevant registers
                struct fence_helper *fh;

                fh = fence_helper_init(NULL, NULL);
                fence_helper_init_reserve(fh, dmabuf, WRITE or READ or DMA);

        just before dma start
                fence_helper_commit_reserve(fh);

        after the completion of dma operation
                fence_helper_signal(fh);

User interfaces.
DMA_BUF_GET_FENCE - this ioctl command performs the above 1-3 steps of
interfaces. a fence helper object is created by fence_helper_init() and a
pointer of the fence helper object is passed to user.

DMA_BUF_PUT_FENCE - this ioctl command performs the above 4 step of
interfaces. the pointer made of step 1 is passed into kernel through this
ioctl command.

And tutorial for user process.
        just before cpu access
                struct dma_buf_fence *df;

                df->type = DMA_BUF_ACCESS_READ or DMA_BUF_ACCESS_WRITE;
                ioctl(fd, DMA_BUF_GET_FENCE, &df);

        after memset or memcpy
                ioctl(fd, DMA_BUF_PUT_FENCE, &df);

Resource management.
Basically, a fence helper object is created and released internally;
created at fence_helper_init() and released at fence_helper_signal(). And
also this can be done case by case like below,
- If there was any thread blocked, its own resource is released at
fence_helper_commit_reserve() after the thread was waked up.
- If a blocked thread was timed out without fence_helper_singal() call;
anyone hasn't signaled a shared buffer, its own resource is released at
fence_helper_commit_reserve().
- if a shared buffer access by a thread was completed without blocked, its
own resource is released at fence_helper_signal().

References.
[1] http://lists.freedesktop.org/archives/dri-devel/2013-January/033436.html
[2] http://lists.freedesktop.org/archives/dri-devel/2013-January/033434.html

Thanks,
Inki Dae
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/dri-devel/attachments/20130509/c2d57955/attachment.html>