[PATCH V6] drm/xe/mmap: Add mmap support for PCI memory barrier

[Upadhyay, Tejas]

> -----Original Message-----
> From: Auld, Matthew <matthew.auld at intel.com>
> Sent: Wednesday, October 30, 2024 2:29 PM
> To: Upadhyay, Tejas <tejas.upadhyay at intel.com>; intel-
> xe at lists.freedesktop.org
> Cc: Mrozek, Michal <michal.mrozek at intel.com>
> Subject: Re: [PATCH V6] drm/xe/mmap: Add mmap support for PCI memory
> barrier
> 
> On 25/10/2024 12:05, Tejas Upadhyay wrote:
> > In order to avoid having userspace to use MI_MEM_FENCE, we are adding
> > a mechanism for userspace to generate a PCI memory barrier with low
> > overhead (avoiding IOCTL call as well as writing to VRAM will adds
> > some overhead).
> >
> > This is implemented by memory-mapping a page as uncached that is
> > backed by MMIO on the dGPU and thus allowing userspace to do memory
> > write to the page without invoking an IOCTL.
> > We are selecting the MMIO so that it is not accessible from the PCI
> > bus so that the MMIO writes themselves are ignored, but the PCI memory
> > barrier will still take action as the MMIO filtering will happen after
> > the memory barrier effect.
> >
> > When we detect special defined offset in mmap(), We are mapping 4K
> > page which contains the last of page of doorbell MMIO range to
> > userspace for same purpose.
> >
> > For user to query special offset we are adding special flag in
> > mmap_offset ioctl which needs to be passed as follows, struct
> > drm_xe_gem_mmap_offset mmo = {
> >          .handle = 0, /* this must be 0 */
> >          .flags = DRM_XE_MMAP_OFFSET_FLAG_PCI_BARRIER,
> > };
> > igt_ioctl(fd, DRM_IOCTL_XE_GEM_MMAP_OFFSET, &mmo); map =
> mmap(NULL,
> > size, PROT_WRITE, MAP_SHARED, fd, mmo);
> >
> > Note: Test coverage for this is added by IGT
> >        https://patchwork.freedesktop.org/series/140368/  here.
> >        UMD implementing test, once PR is ready will attach with
> >        this patch.
> >
> > V6(MAuld)
> >    - Move physical mmap to fault handler
> >    - Modify kernel-doc and attach UMD PR when ready
> > V5(MAuld)
> >    - Return invalid early in case of non 4K PAGE_SIZE
> >    - Format kernel-doc and add note for 4K PAGE_SIZE HW limit
> > V4(MAuld)
> >    - Add kernel-doc for uapi change
> >    - Restrict page size to 4K
> > V3(MAuld)
> >    - Remove offset defination from UAPI to be able to change later
> >    - Edit commit message for special flag addition
> > V2(MAuld)
> >    - Add fault handler with dummy page to handle unplug device
> >    - Add Build check for special offset to be below normal start page
> >    - Test d3hot, mapping seems to be valid in d3hot as well
> >    - Add more info to commit message

UMD implementation PR to consume this is https://github.com/intel/compute-runtime/pull/772

Tejas

> >
> > Cc: Matthew Auld <matthew.auld at intel.com>
> > Cc: Michal Mrozek <michal.mrozek at intel.com>
> > Signed-off-by: Tejas Upadhyay <tejas.upadhyay at intel.com>
> 
> Reviewed-by: Matthew Auld <matthew.auld at intel.com>