[Intel-gfx] [PATCH] drm/i915: Disable atomics in L3 for gen9

[Tvrtko Ursulin]

On 20/07/2019 15:31, Chris Wilson wrote:
> Enabling atomic operations in L3 leads to unrecoverable GPU hangs, as
> the machine stops responding milliseconds after receipt of the reset
> request [GDRT]. By disabling the cached atomics, the hang do not occur
> and we presume the GPU would reset normally for similar hangs.
> 
> Reported-by: Jason Ekstrand <jason at jlekstrand.net>
> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=110998
> Signed-off-by: Chris Wilson <chris at chris-wilson.co.uk>
> Cc: Jason Ekstrand <jason at jlekstrand.net>
> Cc: Mika Kuoppala <mika.kuoppala at linux.intel.com>
> Cc: Tvrtko Ursulin <tvrtko.ursulin at linux.intel.com>
> ---
> Jason reports that Windows is not clearing L3SQCREG4:22 and does not
> suffer the same GPU hang so it is likely some other w/a that interacts
> badly. Fwiw, these 3 are the only registers I could find that mention
> atomic ops (and appear to be part of the same chain for memory access).

Bit-toggling itself looks fine to me and matches what I could find in 
the docs. (All three bits across three registers should be equal.)

What I am curious about is what are the other consequences of disabling 
L3 atomics? Performance drop somewhere?

Regards,

Tvrtko


> ---
>   drivers/gpu/drm/i915/gt/intel_workarounds.c | 8 ++++++++
>   drivers/gpu/drm/i915/i915_reg.h             | 7 +++++++
>   2 files changed, 15 insertions(+)
> 
> diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c
> index 704ace01e7f5..ac94ed3ba7b6 100644
> --- a/drivers/gpu/drm/i915/gt/intel_workarounds.c
> +++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c
> @@ -1349,6 +1349,14 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
>   		wa_write_or(wal,
>   			    GEN8_L3SQCREG4,
>   			    GEN8_LQSC_FLUSH_COHERENT_LINES);
> +
> +		/* Disable atomics in L3 to prevent unrecoverable hangs */
> +		wa_write_masked_or(wal, GEN9_SCRATCH_LNCF1,
> +				   GEN9_LNCF_NONIA_COHERENT_ATOMICS_ENABLE, 0);
> +		wa_write_masked_or(wal, GEN8_L3SQCREG4,
> +				   GEN8_LQSQ_NONIA_COHERENT_ATOMICS_ENABLE, 0);
> +		wa_write_masked_or(wal, GEN9_SCRATCH1,
> +				   EVICTION_PERF_FIX_ENABLE, 0);
>   	}
>   }
>   
> diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
> index 24f2a52a2b42..e23b2200e7fc 100644
> --- a/drivers/gpu/drm/i915/i915_reg.h
> +++ b/drivers/gpu/drm/i915/i915_reg.h
> @@ -7728,6 +7728,7 @@ enum {
>   #define  GEN11_LQSC_CLEAN_EVICT_DISABLE		(1 << 6)
>   #define  GEN8_LQSC_RO_PERF_DIS			(1 << 27)
>   #define  GEN8_LQSC_FLUSH_COHERENT_LINES		(1 << 21)
> +#define  GEN8_LQSQ_NONIA_COHERENT_ATOMICS_ENABLE REG_BIT(22)
>   
>   /* GEN8 chicken */
>   #define HDC_CHICKEN0				_MMIO(0x7300)
> @@ -11202,6 +11203,12 @@ enum skl_power_gate {
>   /* Media decoder 2 MOCS registers */
>   #define GEN11_MFX2_MOCS(i)	_MMIO(0x10000 + (i) * 4)
>   
> +#define GEN9_SCRATCH_LNCF1		_MMIO(0xb008)
> +#define   GEN9_LNCF_NONIA_COHERENT_ATOMICS_ENABLE REG_BIT(0)
> +
> +#define GEN9_SCRATCH1			_MMIO(0xb11c)
> +#define   EVICTION_PERF_FIX_ENABLE	REG_BIT(8)
> +
>   #define GEN10_SCRATCH_LNCF2		_MMIO(0xb0a0)
>   #define   PMFLUSHDONE_LNICRSDROP	(1 << 20)
>   #define   PMFLUSH_GAPL3UNBLOCK		(1 << 21)
>