[PATCH v2] drm/xe: Support for mmap-ing mmio regions
Matthew Brost
matthew.brost at intel.com
Tue Jun 10 14:21:03 UTC 2025
On Tue, Jun 10, 2025 at 04:57:51PM +0300, Levi, Ilia wrote:
> On 09/06/2025 18:46, Matthew Brost wrote:
> > On Mon, Jun 09, 2025 at 12:59:38PM +0300, Ilia Levi wrote:
> >> Allow the driver to expose hardware register spaces to userspace
> >> through GEM objects with fake mmap offsets. This can be useful
> >> for userspace-firmware communication, debugging, etc.
> >>
> >> v2: Minor doc fix (CI)
> >>
> >> Signed-off-by: Ilia Levi <ilia.levi at intel.com>
> >> ---
> >> drivers/gpu/drm/xe/xe_device_types.h | 14 +++
> >> drivers/gpu/drm/xe/xe_mmio.c | 142 +++++++++++++++++++++++++++
> >> drivers/gpu/drm/xe/xe_mmio.h | 4 +
> >> 3 files changed, 160 insertions(+)
> >>
> >> diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
> >> index ac27389ccb8b..78542de0d48d 100644
> >> --- a/drivers/gpu/drm/xe/xe_device_types.h
> >> +++ b/drivers/gpu/drm/xe/xe_device_types.h
> >> @@ -10,6 +10,7 @@
> >>
> >> #include <drm/drm_device.h>
> >> #include <drm/drm_file.h>
> >> +#include <drm/drm_gem.h>
> >> #include <drm/drm_pagemap.h>
> >> #include <drm/ttm/ttm_device.h>
> >>
> >> @@ -161,6 +162,19 @@ struct xe_mmio {
> >> u32 adj_offset;
> >> };
> >>
> >> +/**
> >> + * struct xe_mmio_gem - GEM wrapper for xe_mmio
> >> + *
> >> + * A GEM object for exposing xe_mmio instance to userspace via mmap.
> >> + */
> >> +struct xe_mmio_gem {
> >> + /** @base: GEM object base */
> >> + struct drm_gem_object base;
> >> +
> >> + /** @mmio: The MMIO region to expose */
> >> + struct xe_mmio mmio;
> > Any reason not this is not a pointer to xe_mmio?
>
>
> Yes, I'd like to keep xe_mmio a lightweight object that could even be defined on the stack.
> Here's an example of how this could be used:
>
> int foo_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
> {
> struct xe_mmio mmio = {0};
> struct xe_mmio_gem *gem;
>
> xe_mmio_init(&mmio, xe_device_get_root_tile(xe), xe->mmio.regs +
> SOME_HW_BLOCK_PCI_OFFSET, SOME_HW_BLOCK_SIZE);
>
> gem = xe_mmio_gem_create(&mmio, file);
> / * store the gem somewhere, return drm_vma_node_offset_addr(&gem->base.vma_node) to the user */
> }
>
> - Ilia
>
>
> >
> >> +};
> >> +
> >> /**
> >> * struct xe_tile - hardware tile structure
> >> *
> >> diff --git a/drivers/gpu/drm/xe/xe_mmio.c b/drivers/gpu/drm/xe/xe_mmio.c
> >> index 7357458bc0d2..6bfa915a9602 100644
> >> --- a/drivers/gpu/drm/xe/xe_mmio.c
> >> +++ b/drivers/gpu/drm/xe/xe_mmio.c
> >> @@ -408,3 +408,145 @@ int xe_mmio_wait32_not(struct xe_mmio *mmio, struct xe_reg reg, u32 mask, u32 va
> >> {
> >> return __xe_mmio_wait32(mmio, reg, mask, val, timeout_us, out_val, atomic, false);
> >> }
> >> +
> >> +/**
> >> + * DOC: Exposing MMIO regions to userspace
> >> + *
> >> + * In certain cases, the driver may allow userspace to mmap a portion of the hardware registers.
> >> + *
> >> + * This can be done as follows:
> >> + * 1. Define an xe_mmio instance that represents this portion.
> >> + * 2. Call xe_mmio_gem_create() to create a GEM object with an mmap-able fake offset.
> >> + * 3. Use drm_vma_node_offset_addr() on the created GEM object to retrieve the fake offset.
> >> + * 4. Provide the fake offset to userspace.
> >> + * 5. Userspace can call mmap with the fake offset. The length provided to mmap
> >> + * must match the size of the xe_mmio instance.
> >> + * 6. When the region is no longer needed, call xe_mmio_gem_destroy() to release the GEM object.
> >> + *
> >> + * Limitations: The exposed xe_mmio must be page-aligned with regards to its BAR offset and size.
> >> + *
> >> + * WARNING: Exposing MMIO regions to userspace can have security and stability implications.
> >> + * Make sure not to expose any sensitive registers.
> > This is no secondary issue, it is a primary one. There is no way we can
> > expose entire PCIe MMIO register space to a non-root user (a root user
> > itself can just mmap the PCIe MMIO bar bypassing the XeKMD if it wants
> > btw).
> >
> > What is the requirement for this patch? I know on other PCIe devices
> > I've worked on we had similar issues where we needed to expose a subset
> > of the PCIe MMIO register space to user space for kernel bypass
> > submission - everything we needed to expose in that case was 4k aligned
> > + tied to a single process via a KMD config before mapping to user
> > space. I suspect to do anything safely here, somethine similar needs to
> > be done.
> >
> > Matt
>
>
> As you've figured out - the idea is of course to expose small chunks (say 1 page)
> that are allocated solely to a specific user. One use-case is direct submission in
> upcoming platforms. I am checking 4K alignment in the patch. Not sure what
> you mean about KMD config - xe_mmio_gem_create accepts a drm_file *.
On the previous device I worked on we'd config some non-user exposed
MMIO registers to tie the exposed user registers to specifc user process
(e.g. we'd setup the PASID of the exposed submission queue). Not sure if
something like is required on the upcoming device(s).
> So the driver performs exposure of an MMIO region on behalf of the user and
> only this user will be able to perform mmap. Do you think anything else is required
> here?
No, I think it makes sense now but personally I'd drop the reuse of
xe_mmio and keep that as a heavy weight kernel object only used in
xe_device, xe_tile, xe_gt, etc...
Can you just pass in size / MMIO address (result of xe_mmio_phys_addr)
to xe_mmio_gem_create? I think that would avoid a bit confusion.
Matt
> - Ilia
>
> >
> >> + */
> >> +
> >> +static void xe_mmio_gem_free(struct drm_gem_object *);
> >> +static int xe_mmio_gem_mmap(struct drm_gem_object *, struct vm_area_struct *);
> >> +
> >> +static const struct vm_operations_struct vm_ops = {
> >> + .open = drm_gem_vm_open,
> >> + .close = drm_gem_vm_close,
> >> +};
> >> +
> >> +static const struct drm_gem_object_funcs xe_mmio_gem_funcs = {
> >> + .free = xe_mmio_gem_free,
> >> + .mmap = xe_mmio_gem_mmap,
> >> + .vm_ops = &vm_ops,
> >> +};
> >> +
> >> +static inline struct xe_mmio_gem *to_xe_mmio_gem(struct drm_gem_object *obj)
> >> +{
> >> + return container_of(obj, struct xe_mmio_gem, base);
> >> +}
> >> +
> >> +static inline phys_addr_t xe_mmio_phys_addr(struct xe_mmio *mmio)
> >> +{
> >> + struct xe_device *xe = tile_to_xe(mmio->tile);
> >> +
> >> + /*
> >> + * All MMIO instances are currently on PCI BAR 0, so we can do the trick below.
> >> + * In the future we may want to store the physical address in struct xe_mmio.
> >> + */
> >> + return pci_resource_start(to_pci_dev(xe->drm.dev), GTTMMADR_BAR) +
> >> + (uintptr_t)(mmio->regs - xe->mmio.regs);
> >> +}
> >> +
> >> +/**
> >> + * xe_mmio_gem_create - Expose an MMIO region to userspace
> >> + * @mmio: xe_mmio instance
> >> + * @file: DRM file descriptor
> >> + *
> >> + * This function creates a GEM object with an mmap-able fake offset that wraps
> >> + * the provided xe_mmio instance.
> >> + *
> >> + * See: "Exposing MMIO regions to userspace"
> >> + */
> >> +struct xe_mmio_gem *
> >> +xe_mmio_gem_create(struct xe_mmio *mmio, struct drm_file *file)
> >> +{
> >> + struct xe_device *xe = tile_to_xe(mmio->tile);
> >> + size_t size = mmio->regs_size;
> >> + struct xe_mmio_gem *obj;
> >> + struct drm_gem_object *base;
> >> + int err;
> >> +
> >> + if ((xe_mmio_phys_addr(mmio) % PAGE_SIZE != 0) || (size % PAGE_SIZE != 0))
> >> + return ERR_PTR(-EINVAL);
> >> +
> >> + obj = kzalloc(sizeof(*obj), GFP_KERNEL);
> >> + if (!obj)
> >> + return ERR_PTR(-ENOMEM);
> >> +
> >> + base = &obj->base;
> >> + base->funcs = &xe_mmio_gem_funcs;
> >> + obj->mmio = *mmio;
> >> +
> >> + drm_gem_private_object_init(&xe->drm, base, size);
> >> +
> >> + err = drm_gem_create_mmap_offset(base);
> >> + if (err)
> >> + goto free_gem;
> >> +
> >> + err = drm_vma_node_allow(&base->vma_node, file);
> >> + if (err)
> >> + goto free_gem;
> >> +
> >> + return obj;
> >> +
> >> +free_gem:
> >> + xe_mmio_gem_free(base);
> >> + return ERR_PTR(err);
> >> +}
> >> +
> >> +static void xe_mmio_gem_free(struct drm_gem_object *base)
> >> +{
> >> + struct xe_mmio_gem *obj = to_xe_mmio_gem(base);
> >> +
> >> + drm_gem_object_release(base);
> >> + kfree(obj);
> >> +}
> >> +
> >> +/**
> >> + * xe_mmio_gem_destroy - Destroy the GEM object wrapping xe_mmio
> >> + * @gem: the GEM object to destroy
> >> + *
> >> + * This function releases resources associated with the GEM object created by
> >> + * xe_mmio_gem_create().
> >> + *
> >> + * See: "Exposing MMIO regions to userspace"
> >> + */
> >> +void xe_mmio_gem_destroy(struct xe_mmio_gem *gem)
> >> +{
> >> + xe_mmio_gem_free(&gem->base);
> >> +}
> >> +
> >> +static int xe_mmio_gem_mmap(struct drm_gem_object *base, struct vm_area_struct *vma)
> >> +{
> >> + struct xe_mmio_gem *obj = to_xe_mmio_gem(base);
> >> + struct xe_mmio *mmio = &obj->mmio;
> >> +
> >> + if (vma->vm_end - vma->vm_start != base->size)
> >> + return -EINVAL;
> >> +
> >> + /*
> >> + * Set vm_pgoff (used as a fake buffer offset by DRM) to 0 and map the
> >> + * whole buffer from the start.
> >> + */
> >> + vma->vm_pgoff = 0;
> >> + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
> >> +
> >> + vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP |
> >> + VM_DONTCOPY | VM_NORESERVE);
> >> +
> >> + return remap_pfn_range(vma, vma->vm_start, xe_mmio_phys_addr(mmio) >> PAGE_SHIFT,
> >> + base->size, vma->vm_page_prot);
> >> +}
> >> diff --git a/drivers/gpu/drm/xe/xe_mmio.h b/drivers/gpu/drm/xe/xe_mmio.h
> >> index c151ba569003..2990bbcef24d 100644
> >> --- a/drivers/gpu/drm/xe/xe_mmio.h
> >> +++ b/drivers/gpu/drm/xe/xe_mmio.h
> >> @@ -8,6 +8,7 @@
> >>
> >> #include "xe_gt_types.h"
> >>
> >> +struct drm_file;
> >> struct xe_device;
> >> struct xe_reg;
> >>
> >> @@ -42,4 +43,7 @@ static inline struct xe_mmio *xe_root_tile_mmio(struct xe_device *xe)
> >> return &xe->tiles[0].mmio;
> >> }
> >>
> >> +struct xe_mmio_gem *xe_mmio_gem_create(struct xe_mmio *mmio, struct drm_file *file);
> >> +void xe_mmio_gem_destroy(struct xe_mmio_gem *gem);
> >> +
> >> #endif
> >> --
> >> 2.43.0
> >>
>
More information about the Intel-xe
mailing list