[PATCH] drm/amdgpu: move GTT allocation from gmc_sw_init to gmc_hw_init

[Liu, Aaron]

[Public]

Guchun.
Thanks your reminder. I need to modify again.

--
Best Regards
Aaron Liu

> -----Original Message-----
> From: Chen, Guchun <Guchun.Chen at amd.com>
> Sent: Thursday, January 27, 2022 3:10 PM
> To: Liu, Aaron <Aaron.Liu at amd.com>; amd-gfx at lists.freedesktop.org
> Cc: Ji, Ruili <Ruili.Ji at amd.com>; Kim, Jonathan <Jonathan.Kim at amd.com>;
> Kuehling, Felix <Felix.Kuehling at amd.com>; Liu, Aaron
> <Aaron.Liu at amd.com>; Huang, Ray <Ray.Huang at amd.com>; Deucher,
> Alexander <Alexander.Deucher at amd.com>
> Subject: RE: [PATCH] drm/amdgpu: move GTT allocation from gmc_sw_init
> to gmc_hw_init
> 
> [Public]
> 
> This will create sdma_access_bo only for ASIC with gmc v10? Original
> creation occurs in amdgpu_ttm_init, it's not limited to ASICs with gmc v10.
> 
> Regards,
> Guchun
> 
> -----Original Message-----
> From: amd-gfx <amd-gfx-bounces at lists.freedesktop.org> On Behalf Of
> Aaron Liu
> Sent: Thursday, January 27, 2022 3:04 PM
> To: amd-gfx at lists.freedesktop.org
> Cc: Ji, Ruili <Ruili.Ji at amd.com>; Kim, Jonathan <Jonathan.Kim at amd.com>;
> Kuehling, Felix <Felix.Kuehling at amd.com>; Liu, Aaron
> <Aaron.Liu at amd.com>; Huang, Ray <Ray.Huang at amd.com>; Deucher,
> Alexander <Alexander.Deucher at amd.com>
> Subject: [PATCH] drm/amdgpu: move GTT allocation from gmc_sw_init to
> gmc_hw_init
> 
> The below patch causes system hang for harvested ASICs.
> d015e9861e55 drm/amdgpu: improve debug VRAM access performance
> using sdma
> 
> The root cause is that GTT buffer should be allocated after GC SA harvest
> programming completed.
> 
> For harvested AISC, the GC SA harvest process(see utcl2_harvest) is
> programmed in gmc_v10_0_hw_init function. This is a hardware
> programming.
> Therefore should be located in hw init. Hence need to move GTT allocation
> from gmc_v10_0_sw_init to gmc_v10_0_hw_init.
> 
> Signed-off-by: Aaron Liu <aaron.liu at amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 8 --------
> drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c  | 9 +++++++++
>  2 files changed, 9 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> index 3d8a20956b74..7ce0478b2908 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -1823,12 +1823,6 @@ int amdgpu_ttm_init(struct amdgpu_device
> *adev)
>  		return r;
>  	}
> 
> -	if (amdgpu_bo_create_kernel(adev, PAGE_SIZE, PAGE_SIZE,
> -				AMDGPU_GEM_DOMAIN_GTT,
> -				&adev->mman.sdma_access_bo, NULL,
> -				&adev->mman.sdma_access_ptr))
> -		DRM_WARN("Debug VRAM access will use slowpath MM
> access\n");
> -
>  	return 0;
>  }
> 
> @@ -1850,8 +1844,6 @@ void amdgpu_ttm_fini(struct amdgpu_device
> *adev)
>  	if (adev->mman.stolen_reserved_size)
>  		amdgpu_bo_free_kernel(&adev-
> >mman.stolen_reserved_memory,
>  				      NULL, NULL);
> -	amdgpu_bo_free_kernel(&adev->mman.sdma_access_bo, NULL,
> -					&adev->mman.sdma_access_ptr);
>  	amdgpu_ttm_fw_reserve_vram_fini(adev);
> 
>  	if (drm_dev_enter(adev_to_drm(adev), &idx)) { diff --git
> a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> index 73ab0eebe4e2..c560bdc2a93c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> @@ -1062,6 +1062,12 @@ static int gmc_v10_0_hw_init(void *handle)
>  	if (adev->umc.funcs && adev->umc.funcs->init_registers)
>  		adev->umc.funcs->init_registers(adev);
> 
> +	if (amdgpu_bo_create_kernel(adev, PAGE_SIZE, PAGE_SIZE,
> +				AMDGPU_GEM_DOMAIN_GTT,
> +				&adev->mman.sdma_access_bo, NULL,
> +				&adev->mman.sdma_access_ptr))
> +		DRM_WARN("Debug VRAM access will use slowpath MM
> access\n");
> +
>  	return 0;
>  }
> 
> @@ -1082,6 +1088,9 @@ static int gmc_v10_0_hw_fini(void *handle)  {
>  	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> 
> +	amdgpu_bo_free_kernel(&adev->mman.sdma_access_bo, NULL,
> +					&adev->mman.sdma_access_ptr);
> +
>  	gmc_v10_0_gart_disable(adev);
> 
>  	if (amdgpu_sriov_vf(adev)) {
> --
> 2.25.1