[PATCH 0/6] drm/msm: Add per-instance pagetables

[Jordan Crouse]

Using the framework described here

https://lists.linuxfoundation.org/pipermail/iommu/2017-March/020716.html

This implements per-instance pagetables for the GPU driver creating an
individual pagetable for each file descriptor (so not strictly per-process
but in practice we can't share buffers between file descriptors anyway without
importing them).

This is the brief workflow for the process:

 - At init, the driver attaches an UNMANGED domain to the IOMMU (context bank 0)

 - All "global" buffers (kernel side GPU buffers such as ringbuffers, etc) are
   mapped into the TTBR1 space which is defined as any address with bit 48 set.
   In pratice we have discovered that for reasons yet uknown, bit 47 also has
   to be set for the GPU to sign extend correctly, so the TTBR1 region is
   defined as starting at 0xffff8_0000_0000_0000.

 - When a new file descriptor is opened, a dynamic domain is cloned from the
   real domain - this does not program the hardware but it creates a pagetable
   and returns a pointer that we can use to map memory to - this is wrapped in a
   new addresss space and used for all allocations created with the file
   descriptor.

 - At command submission time, a SMMU_TABLE_UPDATE packet is set before every
   command which contains the physical address of the TTBR0 register for the
   pagetable associated with the process - the GPU will automatically switch
   the pagetable for the process.

Because no kernel side allocations are in the TTBR0 space there is no setup
required to switch the TTBR0 pagetable and we do not need to reprogram it
after the command is over since the next command will rewrite the register.
This makes the code significantly more simple than it could be (*cough*
downstream *cough*).

I'm sure there will be questions, and I'm sure that what we have won't be what
is finally decided upon in the arm-smmu driver (in particular there are some
nice parts of the arm-v3 SVM solution that we can borrow) but I think it is
important to get eyeballs on this for posterity.

Thanks!
Jordan

Jordan Crouse (6):
  drm/msm: Enable 64 bit mode by default
  drm/msm: Pass the MMU domain index in struct msm_file_private
  drm/msm: Make separate iommu function tables for v1 and v2 MMUs
  drm/msm: Use TTBR1 for kernel side GPU buffer objects
  drm/msm: Support dynamic IOMMU domains
  drm/msm: a5xx: Support per-instance pagetables

 arch/arm64/boot/dts/qcom/msm8996.dtsi     |   2 +
 drivers/gpu/drm/msm/adreno/a5xx_gpu.c     |  78 ++++++++++++++-
 drivers/gpu/drm/msm/adreno/a5xx_gpu.h     |  17 ++++
 drivers/gpu/drm/msm/adreno/a5xx_preempt.c |  61 +++++++++---
 drivers/gpu/drm/msm/adreno/adreno_gpu.c   |  18 +++-
 drivers/gpu/drm/msm/adreno/adreno_gpu.h   |   2 +
 drivers/gpu/drm/msm/msm_drv.c             |  60 +++++++++---
 drivers/gpu/drm/msm/msm_drv.h             |   9 +-
 drivers/gpu/drm/msm/msm_gem.h             |   1 +
 drivers/gpu/drm/msm/msm_gem_submit.c      |  12 ++-
 drivers/gpu/drm/msm/msm_gem_vma.c         |  38 ++++++--
 drivers/gpu/drm/msm/msm_gpu.c             |   3 +-
 drivers/gpu/drm/msm/msm_iommu.c           | 151 ++++++++++++++++++++++++------
 drivers/gpu/drm/msm/msm_iommu.h           |  34 +++++++
 drivers/gpu/drm/msm/msm_mmu.h             |   2 +-
 15 files changed, 415 insertions(+), 73 deletions(-)
 create mode 100644 drivers/gpu/drm/msm/msm_iommu.h

-- 
1.9.1