[Mesa-dev] [PATCH 2/4] i965/hsw: Properly handle RO state invalidation

[Ben Widawsky]

Before invalidating the "RO state" caches, we have to emit a PIPE_CONTROL...

The following is the best I can explain things as I understand them without
getting too far into the low-level details.

I believe the "read only" aspect of this is misleading. It just so happens that
the state caches are read only, but really this is about state cache coherency.
As I understand it, when invalidating any of the state caches, the internal
hardware that handles this stuff (state arbiter, sometimes referred to as sarb)
must flush and sync on any pending requests before it can send an invalidation.
Why this is needed is based on the uArch design. Essentially you can have
multiple pending requests for *things* and the invalidation must occur after
those pending things.

The result of this is the following:
   Programming Note: PIPECONTROL with RO Cache Invalidation: Prior to
   programming a PIPECONTROL command with any of the RO cache invalidation bit
   set, program a PIPECONTROL flush command with “CS stall” bit and “HDC Flush”
   bit set.

Unfortunately, there is no such thing as an "HDC Flush" bit anywhere that I can
find. AFAICT, HDC is short for H? Data Cluster. So I've settled on
PIPE_CONTROL_DATA_CACHE_INVALIDATE as the closest sounding thing.

I've put this code before the existing every 4 PIPE_CONTROL workaround because
this might emit a PIPE_CONTROL which would void the need to use that workaround.
It probably makes no difference since back to back CS stalls probably have
little measurable impact anyway - but it would be less commands.

v2: Rebased on master
Updated commit message
Filled picked a different PIPE_CONTROL flag

Signed-off-by: Ben Widawsky <ben at bwidawsk.net>
---
 src/mesa/drivers/dri/i965/intel_batchbuffer.c | 12 ++++++++++++
 src/mesa/drivers/dri/i965/intel_reg.h         |  4 ++++
 2 files changed, 16 insertions(+)

diff --git a/src/mesa/drivers/dri/i965/intel_batchbuffer.c b/src/mesa/drivers/dri/i965/intel_batchbuffer.c
index f19406f..db2f345 100644
--- a/src/mesa/drivers/dri/i965/intel_batchbuffer.c
+++ b/src/mesa/drivers/dri/i965/intel_batchbuffer.c
@@ -453,6 +453,17 @@ gen7_cs_stall_every_four_pipe_controls(struct brw_context *brw, uint32_t flags)
    return 0;
 }
 
+static uint32_t
+hsw_stall_before_ro_state_cache_invalidate(struct brw_context *brw, uint32_t flags)
+{
+      if (brw->is_haswell && (flags & READ_ONLY_CACHE_INVALIDATIONS)) {
+         return PIPE_CONTROL_DATA_CACHE_INVALIDATE |
+                PIPE_CONTROL_CS_STALL;
+      }
+
+      return 0;
+}
+
 /**
  * Emit a PIPE_CONTROL with various flushing flags.
  *
@@ -474,6 +485,7 @@ brw_emit_pipe_control_flush(struct brw_context *brw, uint32_t flags)
       OUT_BATCH(0);
       ADVANCE_BATCH();
    } else if (brw->gen >= 6) {
+      flags |= hsw_stall_before_ro_state_cache_invalidate(brw, flags);
       flags |= gen7_cs_stall_every_four_pipe_controls(brw, flags);
 
       BEGIN_BATCH(5);
diff --git a/src/mesa/drivers/dri/i965/intel_reg.h b/src/mesa/drivers/dri/i965/intel_reg.h
index 488fb5b..44765ad 100644
--- a/src/mesa/drivers/dri/i965/intel_reg.h
+++ b/src/mesa/drivers/dri/i965/intel_reg.h
@@ -78,6 +78,10 @@
 #define PIPE_CONTROL_DEPTH_CACHE_FLUSH		(1 << 0)
 #define PIPE_CONTROL_PPGTT_WRITE	(0 << 2)
 #define PIPE_CONTROL_GLOBAL_GTT_WRITE	(1 << 2)
+#define READ_ONLY_CACHE_INVALIDATIONS   (PIPE_CONTROL_INSTRUCTION_INVALIDATE | \
+                                         PIPE_CONTROL_TEXTURE_CACHE_INVALIDATE | \
+                                         PIPE_CONTROL_CONST_CACHE_INVALIDATE | \
+                                         PIPE_CONTROL_STATE_CACHE_INVALIDATE)
 
 /** @} */
 
-- 
2.3.1