[virglrenderer-devel] coherent memory access for virgl

[Gurchetan Singh]

On Wed, Oct 17, 2018 at 12:18 AM Gerd Hoffmann <kraxel at redhat.com> wrote:
>
>   Hi,
>
> > > Using the pci bar also for non-coherent buffers becomes tricky.  The
> > > buffer is out-of-reach for the guest.  We can ask the host gbm_bo_map()
> > > the buffer and place it in the pci bar for access.  But it's not
> > > permanent (unless we can change libgbm as outlined above).  I'm not sure
> > > how to model that best in ttm ...
> >
> > There are three possible cases:
> >
> > 1) Coherent -- guest can keep the resource mapped (most Intel, ARM
> > display drivers since they primarily use uncached / write_combine
> > memory)
> > 2) Non-coherent, but sync-able (non-LLC Intel platorms, MSM)
> > 3) Non-coherent and no-flush/invalidate mechanism exposed yet (AMD)
> >
> > We should make these realities explicit to guest user space through
> > the flags (i.e, REQUIRES_SYNC, MEMORY_COHERENT etc.).
>
> Hmm, sounds like coherency is more a property of the hardware, not of
> the buffer.  Or does hardware exist which supports both (coherent and
> non-coherent) kinds of buffers, and you have to ask for the coherent
> variant if you need it?

You can have non-coherent and coherent memory on the same machine.
Check out slides 8 --> 11 of this presentation:

https://www.khronos.org/assets/uploads/developers/library/2018-vulkan-devday/03-Memory.pdf

http://www.vulkan.gpuinfo.org/ (Click on a device and choose the memory tab)

Coherence is a property of the mapping.  You can have non-coherent and
coherent views into the same memory (for example, if
dma_sync_sg_for_cpu / dma_sync_sg_for_sync is used, ARM display
drivers can have cached memory).

Vulkan simplifies this by making it an allocation time property, more
specifically the memoryIndex in VkMemoryAllocateInfo:

https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VkMemoryAllocateInfo.html

We can do something similar.  Additionally, I was chatting with
chadversary at + (the author of the recently released
VK_EXT_image_drm_format_modifier / VK_EXT_external_memory_dma_buf
specs) and importing into Vulkan requires the memory type.

External allocators have not caught up yet, but will also have to.

>
> > gbm.h doesn't expose these capabilities yet, but I can imagine a
> > virglrenderer function that queries for this information in response
> > to the resource_info ioctl.
>
> Well, for now I'm looking at the virtio protocol level.  resource_info
> simply doesn't exist there, so that must be tackled first before we can
> figure the best way to tell userspace.
>
> cheers,
>   Gerd
>