[PATCH v2] drm/xe/bmg: improve cache flushing behaviour

[Nirmoy Das]

On 10/7/2024 9:45 AM, Matthew Auld wrote:
> The BSpec says that EN_L3_RW_CCS_CACHE_FLUSH must be toggled
> on for manual global invalidation to take effect and actually flush
> device cache, however this also turns on flushing for things like
> pipecontrol, which occurs between submissions for compute/render. This
> sounds like massive overkill for our needs, where we already have the
> manual flushing on the display side with the global invalidation. Some
> observations on BMG:
>
> 1. Disabling l2 caching for host writes and stubbing out the driver
>    global invalidation but keeping EN_L3_RW_CCS_CACHE_FLUSH enabled, has
>    no impact on wb-transient-vs-display IGT, which makes sense since the
>    pipecontrol is now flushing the device cache after the render copy.
>    Without EN_L3_RW_CCS_CACHE_FLUSH the test then fails, which is also
>    expected since device cache is now dirty and display engine can't see
>    the writes.
>
> 2. Disabling EN_L3_RW_CCS_CACHE_FLUSH, but keeping the driver global
>    invalidation also has no impact on wb-transient-vs-display. This
>    suggests that the global invalidation still works as expected and is
>    flushing the device cache without EN_L3_RW_CCS_CACHE_FLUSH turned on.
>
> With that drop EN_L3_RW_CCS_CACHE_FLUSH. This helps some workloads since
> we no longer flush the device cache between submissions as part of
> pipecontrol.
>
> Edit: We now also have clarification from HW side that BSpec was indeed
> wrong here.
>
> v2:
>   - Rebase and update commit message.
>
> BSpec: 71718
> Signed-off-by: Matthew Auld <matthew.auld at intel.com>
> Cc: Vitasta Wattal <vitasta.wattal at intel.com>
> Cc: Matt Roper <matthew.d.roper at intel.com>
> Cc: Nirmoy Das <nirmoy.das at intel.com>
Reviewed-by: Nirmoy Das <nirmoy.das at intel.com>
> ---
>  drivers/gpu/drm/xe/regs/xe_gt_regs.h | 3 ---
>  drivers/gpu/drm/xe/xe_gt.c           | 1 -
>  2 files changed, 4 deletions(-)
>
> diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
> index fb80042cbe0d..e98b7b91116d 100644
> --- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
> +++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
> @@ -401,9 +401,6 @@
>  
>  #define XE2_GLOBAL_INVAL			XE_REG(0xb404)
>  
> -#define SCRATCH1LPFC				XE_REG(0xb474)
> -#define   EN_L3_RW_CCS_CACHE_FLUSH		REG_BIT(0)
> -
>  #define XE2LPM_L3SQCREG2			XE_REG_MCR(0xb604)
>  
>  #define XE2LPM_L3SQCREG3			XE_REG_MCR(0xb608)
> diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
> index 2d5e78311b76..1c79660fb086 100644
> --- a/drivers/gpu/drm/xe/xe_gt.c
> +++ b/drivers/gpu/drm/xe/xe_gt.c
> @@ -108,7 +108,6 @@ static void xe_gt_enable_host_l2_vram(struct xe_gt *gt)
>  		return;
>  
>  	if (!xe_gt_is_media_type(gt)) {
> -		xe_mmio_write32(&gt->mmio, SCRATCH1LPFC, EN_L3_RW_CCS_CACHE_FLUSH);
>  		reg = xe_gt_mcr_unicast_read_any(gt, XE2_GAMREQSTRM_CTRL);
>  		reg |= CG_DIS_CNTLBUS;
>  		xe_gt_mcr_multicast_write(gt, XE2_GAMREQSTRM_CTRL, reg);