[PATCH v3 8/9] drm/amdgpu/ttm: Allocate/Free 4K MMIO_REMAP Singleton BO

[Alex Deucher]

On Thu, Aug 28, 2025 at 5:28 AM Srinivasan Shanmugam
<srinivasan.shanmugam at amd.com> wrote:
>
> Add amdgpu_ttm_mmio_remap_bo_init()/fini() to manage the kernel-owned
> one-page(4K) MMIO_REMAP BO. The allocator runs during TTM init when the
> hardware exposes a remap base (adev->rmmio_base) and the host PAGE_SIZE
> is <= AMDGPU_GPU_PAGE_SIZE (4K).
>
> The helper is idempotent (returns 0 if already allocated) and only
> returns an error when the actual allocation fails.
>
> This keeps MMIO_REMAP lifetime handling localized and prepares for the
> subsequent patch that exposes a userspace handle.
>
> v2:
>  - Check mmio_remap bus address (adev->rmmio_remap.bus_addr) instead of
>    rmmio_base. (Alex)
>  - Skip quietly if PAGE_SIZE > AMDGPU_GPU_PAGE_SIZE or no bus address
>    (no warn). (Alex)
>  - Use `amdgpu_bo_create()` (not *_kernel) - Only with this The object
>    is stored in adev->mmio_remap.bo and will later be exposed to
>    userspace via a GEM handle. (Christian)
>
> Suggested-by: Christian König <christian.koenig at amd.com>
> Suggested-by: Alex Deucher <alexander.deucher at amd.com>
> Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam at amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 64 +++++++++++++++++++++++++
>  1 file changed, 64 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> index 7822d8969c9f..0e301cab74e0 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -1853,6 +1853,63 @@ static void amdgpu_ttm_pools_fini(struct amdgpu_device *adev)
>         adev->mman.ttm_pools = NULL;
>  }
>
> +/**
> + * amdgpu_ttm_mmio_remap_bo_init - allocate the singleton 4K MMIO_REMAP BO
> + * @adev: amdgpu device
> + *
> + * Allocates a one-page (4K) GEM BO in AMDGPU_GEM_DOMAIN_MMIO_REMAP when the
> + * hardware exposes a remap base (adev->rmmio_remap.bus_addr) and the host
> + * PAGE_SIZE is <= AMDGPU_GPU_PAGE_SIZE (4K). The BO is created as a regular
> + * GEM object (amdgpu_bo_create), then reserved and kmap’ed once to exercise
> + * the io-mem setup path. If prerequisites are not met, this is a no-op.
> + *
> + * Return:
> + *  * 0 on success or intentional skip (feature not present/unsupported)
> + *  * negative errno on allocation failure
> + */
> +static int amdgpu_ttm_mmio_remap_bo_init(struct amdgpu_device *adev)
> +{
> +       int r;
> +       struct amdgpu_bo_param bp = { 0 };
> +       void *kptr;
> +
> +       /* Skip if HW doesn’t expose remap or system PAGE > GPU 4K */
> +       if (!adev->rmmio_remap.bus_addr || PAGE_SIZE > AMDGPU_GPU_PAGE_SIZE)
> +               return 0;
> +
> +       /* Hardware remap page is fixed 4K; skip on larger PAGE_SIZE. */
> +       if (PAGE_SIZE > AMDGPU_GPU_PAGE_SIZE)
> +               return 0;

This can be dropped (as per your later patch).

> +
> +       /* Create exactly one GEM BO in the MMIO_REMAP domain. */
> +       bp.type        = ttm_bo_type_device;          /* userspace-mappable GEM */
> +       bp.size        = AMDGPU_GPU_PAGE_SIZE;        /* 4K */
> +       bp.byte_align  = AMDGPU_GPU_PAGE_SIZE;
> +       bp.domain      = AMDGPU_GEM_DOMAIN_MMIO_REMAP;
> +       bp.flags       = 0;
> +       bp.resv        = NULL;
> +       bp.bo_ptr_size = sizeof(struct amdgpu_bo);
> +
> +       r = amdgpu_bo_create(adev, &bp, &adev->rmmio_remap.bo);
> +       if (r)
> +               return r;
> +
> +       return 0;
> +}
> +
> +/**
> + * amdgpu_ttm_mmio_remap_bo_fini - free the singleton MMIO_REMAP BO
> + * @adev: amdgpu device
> + *
> + * Frees the kernel-owned MMIO_REMAP BO if it was allocated by
> + * amdgpu_ttm_mmio_remap_bo_init().
> + */
> +static void amdgpu_ttm_mmio_remap_bo_fini(struct amdgpu_device *adev)
> +{
> +       amdgpu_bo_unref(&adev->rmmio_remap.bo);
> +       adev->mmio_remap.bo = NULL;
> +}
> +
>  /*
>   * amdgpu_ttm_init - Init the memory management (ttm) as well as various
>   * gtt/vram related fields.
> @@ -2027,6 +2084,11 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
>                 return r;
>         }
>
> +       /* Allocate the singleton MMIO_REMAP BO (4K) if supported */
> +       r = amdgpu_ttm_mmio_remap_bo_init(adev);
> +       if (r)
> +               return r;
> +
>         /* Initialize preemptible memory pool */
>         r = amdgpu_preempt_mgr_init(adev);
>         if (r) {
> @@ -2090,6 +2152,8 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev)
>         amdgpu_bo_free_kernel(&adev->mman.sdma_access_bo, NULL,
>                                         &adev->mman.sdma_access_ptr);
>
> +       /* Drop the singleton MMIO_REMAP BO (if allocated) */

You can drop this comment.

Alex

> +       amdgpu_ttm_mmio_remap_bo_fini(adev);
>         amdgpu_ttm_fw_reserve_vram_fini(adev);
>         amdgpu_ttm_drv_reserve_vram_fini(adev);
>
> --
> 2.34.1
>