[Bug 755072] vaapi: expose memory:DMABuf capsfeature

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=755072

--- Comment #59 from Julien Isorce <julien.isorce at gmail.com> ---
I found why we get EACCES "Permission denied" when trying to use mmap on the
dmabuf fd of the exported vasurface. (gst_memory_map (gstfdmem, READWRITE))

It is because the ioctl call that export the surface has params read only.
Similar as if it was doing open(..., "r").

The following hack allow the mmap RW to succeed:

In git://anongit.freedesktop.org/mesa/drm:

diff --git a/xf86drm.c b/xf86drm.c
index 9cfca49..3dfecf1 100644
--- a/xf86drm.c
+++ b/xf86drm.c
@@ -2737,7 +2737,7 @@ int drmPrimeHandleToFD(int fd, uint32_t handle, uint32_t
flags, int *prime_fd)
     memclear(args);
     args.fd = -1;
     args.handle = handle;
-    args.flags = flags;
+    args.flags = flags | DRM_RDWR;
     ret = drmIoctl(fd, DRM_IOCTL_PRIME_HANDLE_TO_FD, &args);
     if (ret)
         return ret;

But in both intel va driver and gallium va driver, this function is not called
directly but it calls another libdrm function that only set the flag
DRM_CLOEXEC. So readonly by default.

Also in vaAcquireBufferHandle's param there is no way to configure readonly or
readwrite. So I think there is no intention to allow the user to have write
access, only mmap READ is intended to work.

But anyway I did had a go with the above hack and I could have the following
pipeline working:
vaapih264dec ! gamma gamma=5 ! glimagesink since gamma element is working
in-place.

Also be aware that with mmap readwrite on a dmabuf we are supposed to do: 

mmap
struct drm_prime_handle args;
args.flags = DMA_BUF_SYNC_START;
ioctl(dma_buf_fd, DMA_BUF_IOCTL_SYNC, &args

then you can change the data/pixels

struct drm_prime_handle args;
args.flags = DMA_BUF_SYNC_END;
ioctl(dma_buf_fd, DMA_BUF_IOCTL_SYNC, &args
munmap

So I did added that in GstFdMemory::map/unmap but I could not noticed any
different since I guess the pipeline is already taking case of the
synchronization.

Funny enough, I then talked to Tiago Vignatti and they are doing the same hack
for ChromeOS, see
https://chromium.googlesource.com/chromiumos/platform/minigbm/+/b96ffe1983295492bc4b4f28b22d6f0e78af69ef%5E!/#F0

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.