Mesa (master): broadcom/compiler: optimize branch emission for uniform break/continue
GitLab Mirror
gitlab-mirror at kemper.freedesktop.org
Mon Apr 5 06:52:58 UTC 2021
Module: Mesa
Branch: master
Commit: 9ca0e070e74f3379ec9ec711d147f79d1caa9ab5
URL: http://cgit.freedesktop.org/mesa/mesa/commit/?id=9ca0e070e74f3379ec9ec711d147f79d1caa9ab5
Author: Iago Toral Quiroga <itoral at igalia.com>
Date: Wed Mar 31 09:09:45 2021 +0200
broadcom/compiler: optimize branch emission for uniform break/continue
A break/continue in a loop is typically emitted like this:
if (cond) {
break/continue;
} else {
}
If cond is uniform, we'll emit code for a uniform if statement and
that will emit a branch right before the if to jump directly to the
else (or the block after the else in this case, since the else is
empty) in case cond evaluates to false. This means we end up emitting
two consecutive branch instructions, one before the if and one for the
THEN block right after:
branch(!cond) -> jump to else (or after else) if cond is false
nop
nop
nop
branch -> unconditional jump to break/continue
nop
nop
nop
Instead, if we are in this scenario, we can do better by emitting the
conditional jump directly and avoiding the "jump to else" case:
branch(cond) -> jump to break/continue if cond is true
nop
nop
nop
We need to be careful when emitting the break/continue for the case
where all lanes are disabled to avoid infinite loops: if we have a
break we always want to take the jump, but we don't want to take it
if it is a continue.
total instructions in shared programs: 13563672 -> 13557348 (-0.05%)
instructions in affected programs: 348034 -> 341710 (-1.82%)
helped: 1158
HURT: 10
Instructions are helped.
total uniforms in shared programs: 3779137 -> 3777535 (-0.04%)
uniforms in affected programs: 90583 -> 88981 (-1.77%)
helped: 1169
HURT: 0
Uniforms are helped.
total max-temps in shared programs: 2317670 -> 2317575 (<.01%)
max-temps in affected programs: 1943 -> 1848 (-4.89%)
helped: 85
HURT: 4
Max-temps are helped.
total sfu-stalls in shared programs: 32247 -> 32247 (0.00%)
sfu-stalls in affected programs: 69 -> 69 (0.00%)
helped: 7
HURT: 9
Inconclusive result (value mean confidence interval includes 0).
total inst-and-stalls in shared programs: 13595919 -> 13589595 (-0.05%)
inst-and-stalls in affected programs: 350674 -> 344350 (-1.80%)
helped: 1154
HURT: 11
Inst-and-stalls are helped.
total nops in shared programs: 358202 -> 354325 (-1.08%)
nops in affected programs: 17367 -> 13490 (-22.32%)
helped: 1168
HURT: 1
Nops are helped.
Reviewed-by: Alejandro Piñeiro <apinheiro at igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9948>
---
src/broadcom/compiler/nir_to_vir.c | 94 ++++++++++++++++++++++++++++----------
1 file changed, 70 insertions(+), 24 deletions(-)
diff --git a/src/broadcom/compiler/nir_to_vir.c b/src/broadcom/compiler/nir_to_vir.c
index 44a326f606c..7135f2ce761 100644
--- a/src/broadcom/compiler/nir_to_vir.c
+++ b/src/broadcom/compiler/nir_to_vir.c
@@ -3158,37 +3158,83 @@ ntq_emit_uniform_if(struct v3d_compile *c, nir_if *if_stmt)
else
else_block = vir_new_block(c);
+ /* Check if this if statement is really just a conditional jump with
+ * the form:
+ *
+ * if (cond) {
+ * break/continue;
+ * } else {
+ * }
+ *
+ * In which case we can skip the jump to ELSE we emit before the THEN
+ * block and instead just emit the break/continue directly.
+ */
+ nir_jump_instr *conditional_jump = NULL;
+ if (empty_else_block) {
+ nir_block *nir_then_block = nir_if_first_then_block(if_stmt);
+ struct nir_instr *inst = nir_block_first_instr(nir_then_block);
+ if (inst && inst->type == nir_instr_type_jump)
+ conditional_jump = nir_instr_as_jump(inst);
+ }
+
/* Set up the flags for the IF condition (taking the THEN branch). */
enum v3d_qpu_cond cond = ntq_emit_bool_to_cond(c, if_stmt->condition);
- /* Jump to ELSE. */
- struct qinst *branch = vir_BRANCH(c, cond == V3D_QPU_COND_IFA ?
- V3D_QPU_BRANCH_COND_ANYNA :
- V3D_QPU_BRANCH_COND_ANYA);
- /* Pixels that were not dispatched or have been discarded should not
- * contribute to the ANYA/ANYNA condition.
- */
- branch->qpu.branch.msfign = V3D_QPU_MSFIGN_P;
+ if (!conditional_jump) {
+ /* Jump to ELSE. */
+ struct qinst *branch = vir_BRANCH(c, cond == V3D_QPU_COND_IFA ?
+ V3D_QPU_BRANCH_COND_ANYNA :
+ V3D_QPU_BRANCH_COND_ANYA);
+ /* Pixels that were not dispatched or have been discarded
+ * should not contribute to the ANYA/ANYNA condition.
+ */
+ branch->qpu.branch.msfign = V3D_QPU_MSFIGN_P;
- vir_link_blocks(c->cur_block, else_block);
- vir_link_blocks(c->cur_block, then_block);
+ vir_link_blocks(c->cur_block, else_block);
+ vir_link_blocks(c->cur_block, then_block);
- /* Process the THEN block. */
- vir_set_emit_block(c, then_block);
- ntq_emit_cf_list(c, &if_stmt->then_list);
+ /* Process the THEN block. */
+ vir_set_emit_block(c, then_block);
+ ntq_emit_cf_list(c, &if_stmt->then_list);
- if (!empty_else_block) {
- /* At the end of the THEN block, jump to ENDIF, unless
- * the block ended in a break or continue.
- */
- if (!c->cur_block->branch_emitted) {
- vir_BRANCH(c, V3D_QPU_BRANCH_COND_ALWAYS);
- vir_link_blocks(c->cur_block, after_block);
- }
+ if (!empty_else_block) {
+ /* At the end of the THEN block, jump to ENDIF, unless
+ * the block ended in a break or continue.
+ */
+ if (!c->cur_block->branch_emitted) {
+ vir_BRANCH(c, V3D_QPU_BRANCH_COND_ALWAYS);
+ vir_link_blocks(c->cur_block, after_block);
+ }
- /* Emit the else block. */
- vir_set_emit_block(c, else_block);
- ntq_emit_cf_list(c, &if_stmt->else_list);
+ /* Emit the else block. */
+ vir_set_emit_block(c, else_block);
+ ntq_emit_cf_list(c, &if_stmt->else_list);
+ }
+ } else {
+ /* Emit the conditional jump directly.
+ *
+ * Use ALL with breaks and ANY with continues to ensure that
+ * we always break and never continue when all lanes have been
+ * disabled (for example because of discards) to prevent
+ * infinite loops.
+ */
+ assert(conditional_jump &&
+ (conditional_jump->type == nir_jump_continue ||
+ conditional_jump->type == nir_jump_break));
+
+ struct qinst *branch = vir_BRANCH(c, cond == V3D_QPU_COND_IFA ?
+ (conditional_jump->type == nir_jump_break ?
+ V3D_QPU_BRANCH_COND_ALLA :
+ V3D_QPU_BRANCH_COND_ANYA) :
+ (conditional_jump->type == nir_jump_break ?
+ V3D_QPU_BRANCH_COND_ALLNA :
+ V3D_QPU_BRANCH_COND_ANYNA));
+ branch->qpu.branch.msfign = V3D_QPU_MSFIGN_P;
+
+ vir_link_blocks(c->cur_block,
+ conditional_jump->type == nir_jump_break ?
+ c->loop_break_block :
+ c->loop_cont_block);
}
vir_link_blocks(c->cur_block, after_block);
More information about the mesa-commit
mailing list