Mesa (staging/22.0): r300: guard for unsigned underflow when unrolling loops

[GitLab Mirror]

Module: Mesa
Branch: staging/22.0
Commit: 0844c6c618c2864e9ab608b629a662e5203d6b11
URL:    http://cgit.freedesktop.org/mesa/mesa/commit/?id=0844c6c618c2864e9ab608b629a662e5203d6b11

Author: Pavel Ondračka <pavel.ondracka at gmail.com>
Date:   Tue Feb  8 15:46:36 2022 +0100

r300: guard for unsigned underflow when unrolling loops

If we by some chance end with more instructions than the maximum
amount we can handle, for example from previous branch lowering,
we would underflow while calculating the number of unrolling
iterations and unroll till OOM.

Fixes OOM in gnome-shell 42

Signed-off-by: Pavel Ondračka <pavel.ondracka at gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14992>

---

 src/gallium/drivers/r300/compiler/radeon_emulate_loops.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/gallium/drivers/r300/compiler/radeon_emulate_loops.c b/src/gallium/drivers/r300/compiler/radeon_emulate_loops.c
index ef627b95ca5..c3445a50bfd 100644
--- a/src/gallium/drivers/r300/compiler/radeon_emulate_loops.c
+++ b/src/gallium/drivers/r300/compiler/radeon_emulate_loops.c
@@ -61,6 +61,8 @@ static unsigned int loop_max_possible_iterations(struct radeon_compiler *c,
 {
 	unsigned int total_i = rc_recompute_ips(c);
 	unsigned int loop_i = (loop->EndLoop->IP - loop->BeginLoop->IP) - 1;
+	if(total_i > c->max_alu_insts)
+		return 1;
 	/* +1 because the program already has one iteration of the loop. */
 	return 1 + ((c->max_alu_insts - total_i) / loop_i);
 }