Mesa (main): aco: treat flat-like as vmem in some scheduling heuristics

[GitLab Mirror]

Module: Mesa
Branch: main
Commit: 97e9e42e0d9c04b8c44fa1430366171eda6f5790
URL:    http://cgit.freedesktop.org/mesa/mesa/commit/?id=97e9e42e0d9c04b8c44fa1430366171eda6f5790

Author: Rhys Perry <pendingchaos02 at gmail.com>
Date:   Thu May 19 18:21:34 2022 +0100

aco: treat flat-like as vmem in some scheduling heuristics

fossil-db (navi21):
Totals from 12 (0.01% of 162293) affected shaders:
Instrs: 48754 -> 48762 (+0.02%)
CodeSize: 267092 -> 267124 (+0.01%)
Latency: 1293798 -> 1292303 (-0.12%); split: -0.12%, +0.00%
InvThroughput: 854599 -> 853578 (-0.12%)
VClause: 1623 -> 1619 (-0.25%)
SClause: 1187 -> 1188 (+0.08%); split: -0.08%, +0.17%

fossil-db (vega10):
Totals from 1 (0.00% of 161355) affected shaders:
Latency: 18720 -> 18848 (+0.68%)
InvThroughput: 5775 -> 5776 (+0.02%)
SClause: 12 -> 11 (-8.33%)

Signed-off-by: Rhys Perry <pendingchaos02 at gmail.com>
Reviewed-by: Daniel Schürmann <daniel at schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17079>

---

 src/amd/compiler/aco_scheduler.cpp | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/src/amd/compiler/aco_scheduler.cpp b/src/amd/compiler/aco_scheduler.cpp
index 6fc84fa0e7f..3aeb30c5ccf 100644
--- a/src/amd/compiler/aco_scheduler.cpp
+++ b/src/amd/compiler/aco_scheduler.cpp
@@ -676,8 +676,9 @@ schedule_SMEM(sched_ctx& ctx, Block* block, std::vector<RegisterDemand>& registe
          break;
       /* only move VMEM instructions below descriptor loads. be more aggressive at higher num_waves
        * to help create more vmem clauses */
-      if (candidate->isVMEM() && (cursor.insert_idx - cursor.source_idx > (ctx.num_waves * 4) ||
-                                  current->operands[0].size() == 4))
+      if ((candidate->isVMEM() || candidate->isFlatLike()) &&
+          (cursor.insert_idx - cursor.source_idx > (ctx.num_waves * 4) ||
+           current->operands[0].size() == 4))
          break;
       /* don't move descriptor loads below buffer loads */
       if (candidate->format == Format::SMEM && current->operands[0].size() == 4 &&
@@ -733,7 +734,7 @@ schedule_SMEM(sched_ctx& ctx, Block* block, std::vector<RegisterDemand>& registe
       /* check if candidate depends on current */
       bool is_dependency = !found_dependency && !ctx.mv.upwards_check_deps(up_cursor);
       /* no need to steal from following VMEM instructions */
-      if (is_dependency && candidate->isVMEM())
+      if (is_dependency && (candidate->isVMEM() || candidate->isFlatLike()))
          break;
 
       if (found_dependency) {
@@ -766,7 +767,7 @@ schedule_SMEM(sched_ctx& ctx, Block* block, std::vector<RegisterDemand>& registe
       MoveResult res = ctx.mv.upwards_move(up_cursor);
       if (res == move_fail_ssa || res == move_fail_rar) {
          /* no need to steal from following VMEM instructions */
-         if (res == move_fail_ssa && candidate->isVMEM())
+         if (res == move_fail_ssa && (candidate->isVMEM() || candidate->isFlatLike()))
             break;
          add_to_hazard_query(&hq, candidate.get());
          ctx.mv.upwards_skip(up_cursor);