[PATCH V4 2/2] drm/amdgpu/mes: Add cleaner shader fence address handling in MES for GFX11

[Alex Deucher]

On Mon, Feb 10, 2025 at 2:07 PM Liu, Shaoyun <Shaoyun.Liu at amd.com> wrote:
>
> [AMD Official Use Only - AMD Internal Distribution Only]
>
> Might need to update the  commit message since this change don't use the write-back memory and  the  package is set_hw_res_1.

Fixed locally.  Thanks.

Alex

>
> Other than that , it looks good to me . And I think we need a similar change for gfx12
>
> Regards
> Shaoyun.liu
>
> -----Original Message-----
> From: Deucher, Alexander <Alexander.Deucher at amd.com>
> Sent: Monday, February 10, 2025 11:28 AM
> To: amd-gfx at lists.freedesktop.org
> Cc: SHANMUGAM, SRINIVASAN <SRINIVASAN.SHANMUGAM at amd.com>; cao, lin <lin.cao at amd.com>; Chen, JingWen (Wayne) <JingWen.Chen2 at amd.com>; Koenig, Christian <Christian.Koenig at amd.com>; Deucher, Alexander <Alexander.Deucher at amd.com>; Liu, Shaoyun <Shaoyun.Liu at amd.com>
> Subject: [PATCH V4 2/2] drm/amdgpu/mes: Add cleaner shader fence address handling in MES for GFX11
>
> From: Srinivasan Shanmugam <srinivasan.shanmugam at amd.com>
>
> This commit introduces enhancements to the handling of the cleaner shader fence in the AMDGPU MES driver:
>
> - The MES (Microcode Execution Scheduler) now sends a PM4 packet to the
>   KIQ (Kernel Interface Queue) to request the cleaner shader, ensuring
>   that requests are handled in a controlled manner and avoiding the
>   race conditions.
> - The CP (Compute Processor) firmware has been updated to use a private
>   bus for accessing specific registers, avoiding unnecessary operations
>   that could lead to issues in VF (Virtual Function) mode.
> - The cleaner shader fence memory address is now set correctly in the
>   `mes_set_hw_res_pkt` structure, allowing for proper synchronization of
>   the cleaner shader execution. This is done by calculating the address
>   using the write-back memory base address and the cleaner fence offset.
>
> Cc: lin cao <lin.cao at amd.com>
> Cc: Jingwen Chen <Jingwen.Chen2 at amd.com>
> Cc: Christian König <christian.koenig at amd.com>
> Cc: Alex Deucher <alexander.deucher at amd.com>
> Suggested-by: Shaoyun Liu <shaoyun.liu at amd.com>
> Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam at amd.com>
> Signed-off-by: Alex Deucher <alexander.deucher at amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/mes_v11_0.c | 17 ++++++++++++-----
>  1 file changed, 12 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
> index e862a3febe2b2..661268172dcf6 100644
> --- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
> @@ -754,7 +754,7 @@ static int mes_v11_0_set_hw_resources_1(struct amdgpu_mes *mes)
>         mes_set_hw_res_pkt.header.dwsize = API_FRAME_SIZE_IN_DWORDS;
>         mes_set_hw_res_pkt.enable_mes_info_ctx = 1;
>
> -       ret = amdgpu_bo_create_kernel(adev, size, PAGE_SIZE,
> +       ret = amdgpu_bo_create_kernel(adev, size + AMDGPU_GPU_PAGE_SIZE,
> +PAGE_SIZE,
>                                 AMDGPU_GEM_DOMAIN_VRAM,
>                                 &mes->resource_1,
>                                 &mes->resource_1_gpu_addr,
> @@ -765,7 +765,10 @@ static int mes_v11_0_set_hw_resources_1(struct amdgpu_mes *mes)
>         }
>
>         mes_set_hw_res_pkt.mes_info_ctx_mc_addr = mes->resource_1_gpu_addr;
> -       mes_set_hw_res_pkt.mes_info_ctx_size = mes->resource_1->tbo.base.size;
> +       mes_set_hw_res_pkt.mes_info_ctx_size = size;
> +       mes_set_hw_res_pkt.cleaner_shader_fence_mc_addr =
> +               mes->resource_1_gpu_addr + size;
> +
>         return mes_v11_0_submit_pkt_and_poll_completion(mes,
>                         &mes_set_hw_res_pkt, sizeof(mes_set_hw_res_pkt),
>                         offsetof(union MESAPI_SET_HW_RESOURCES_1, api_status)); @@ -1632,7 +1635,8 @@ static int mes_v11_0_hw_init(struct amdgpu_ip_block *ip_block)
>         if (r)
>                 goto failure;
>
> -       if (amdgpu_sriov_is_mes_info_enable(adev)) {
> +       if (amdgpu_sriov_is_mes_info_enable(adev) ||
> +           adev->gfx.enable_cleaner_shader) {
>                 r = mes_v11_0_set_hw_resources_1(&adev->mes);
>                 if (r) {
>                         DRM_ERROR("failed mes_v11_0_set_hw_resources_1, r=%d\n", r); @@ -1665,10 +1669,13 @@ static int mes_v11_0_hw_init(struct amdgpu_ip_block *ip_block)  static int mes_v11_0_hw_fini(struct amdgpu_ip_block *ip_block)  {
>         struct amdgpu_device *adev = ip_block->adev;
> -       if (amdgpu_sriov_is_mes_info_enable(adev)) {
> +
> +       if (amdgpu_sriov_is_mes_info_enable(adev) ||
> +           adev->gfx.enable_cleaner_shader) {
>                 amdgpu_bo_free_kernel(&adev->mes.resource_1, &adev->mes.resource_1_gpu_addr,
> -                                       &adev->mes.resource_1_addr);
> +                                     &adev->mes.resource_1_addr);
>         }
> +
>         return 0;
>  }
>
> --
> 2.48.1
>