Mesa (master): i965: Implement a CS stall workaround on Broadwell.

[Kenneth Graunke]

Module: Mesa
Branch: master
Commit: e643c7d036d322c2898c9e65e466d75d0c708dc2
URL:    http://cgit.freedesktop.org/mesa/mesa/commit/?id=e643c7d036d322c2898c9e65e466d75d0c708dc2

Author: Kenneth Graunke <kenneth at whitecape.org>
Date:   Sun Jan 26 00:20:21 2014 -0800

i965: Implement a CS stall workaround on Broadwell.

According to the latest documentation, any PIPE_CONTROL with the
"Command Streamer Stall" bit set must also have another bit set,
with five different options:

   - Render Target Cache Flush
   - Depth Cache Flush
   - Stall at Pixel Scoreboard
   - Post-Sync Operation
   - Depth Stall

I chose "Stall at Pixel Scoreboard" since we've used it effectively
in the past, but the choice is fairly arbitrary.

Implementing this in the PIPE_CONTROL emit helpers ensures that the
workaround will always take effect when it ought to.

Apparently, this workaround may be necessary on older hardware as well;
for now I've only added it to Broadwell as it's absolutely necessary
there.  Subsequent patches could add it to older platforms, provided
someone tests it there.

v2: Only flag "Stall at Pixel Scoreboard" when none of the other bits
    are set (suggested by Ian Romanick).

v3: Prefix the function with "gen8" (requested by Eric).

Signed-off-by: Kenneth Graunke <kenneth at whitecape.org>
Reviewed-by: Ian Romanick <ian.d.romanick at intel.com> (v2)
Reviewed-by: Eric Anholt <eric at anholt.net>

---

 src/mesa/drivers/dri/i965/intel_batchbuffer.c |   36 +++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/src/mesa/drivers/dri/i965/intel_batchbuffer.c b/src/mesa/drivers/dri/i965/intel_batchbuffer.c
index 4624268..a06f298 100644
--- a/src/mesa/drivers/dri/i965/intel_batchbuffer.c
+++ b/src/mesa/drivers/dri/i965/intel_batchbuffer.c
@@ -432,6 +432,38 @@ intel_batchbuffer_data(struct brw_context *brw,
 }
 
 /**
+ * According to the latest documentation, any PIPE_CONTROL with the
+ * "Command Streamer Stall" bit set must also have another bit set,
+ * with five different options:
+ *
+ *  - Render Target Cache Flush
+ *  - Depth Cache Flush
+ *  - Stall at Pixel Scoreboard
+ *  - Post-Sync Operation
+ *  - Depth Stall
+ *
+ * I chose "Stall at Pixel Scoreboard" since we've used it effectively
+ * in the past, but the choice is fairly arbitrary.
+ */
+static void
+gen8_add_cs_stall_workaround_bits(uint32_t *flags)
+{
+   uint32_t wa_bits = PIPE_CONTROL_WRITE_FLUSH |
+                      PIPE_CONTROL_DEPTH_CACHE_FLUSH |
+                      PIPE_CONTROL_WRITE_IMMEDIATE |
+                      PIPE_CONTROL_WRITE_DEPTH_COUNT |
+                      PIPE_CONTROL_WRITE_TIMESTAMP |
+                      PIPE_CONTROL_STALL_AT_SCOREBOARD |
+                      PIPE_CONTROL_DEPTH_STALL;
+
+   /* If we're doing a CS stall, and don't already have one of the
+    * workaround bits set, add "Stall at Pixel Scoreboard."
+    */
+   if ((*flags & PIPE_CONTROL_CS_STALL) != 0 && (*flags & wa_bits) == 0)
+      *flags |= PIPE_CONTROL_STALL_AT_SCOREBOARD;
+}
+
+/**
  * Emit a PIPE_CONTROL with various flushing flags.
  *
  * The caller is responsible for deciding what flags are appropriate for the
@@ -441,6 +473,8 @@ void
 brw_emit_pipe_control_flush(struct brw_context *brw, uint32_t flags)
 {
    if (brw->gen >= 8) {
+      gen8_add_cs_stall_workaround_bits(&flags);
+
       BEGIN_BATCH(6);
       OUT_BATCH(_3DSTATE_PIPE_CONTROL | (6 - 2));
       OUT_BATCH(flags);
@@ -481,6 +515,8 @@ brw_emit_pipe_control_write(struct brw_context *brw, uint32_t flags,
                             uint32_t imm_lower, uint32_t imm_upper)
 {
    if (brw->gen >= 8) {
+      gen8_add_cs_stall_workaround_bits(&flags);
+
       BEGIN_BATCH(6);
       OUT_BATCH(_3DSTATE_PIPE_CONTROL | (6 - 2));
       OUT_BATCH(flags);