[Mesa-dev] [PATCH] radeonsi: compute the absolute value before RSQ

[Samuel Pitoiset]

As explained by Nicolai, it seems like D3D always compute the
absolute value while GLSL says that the result of inversesqrt()
is undefined if x <= 0. Using the absolute value looks like safer
especially when the game has been ported from D3D to GL.

This gets rid of the NaN values in the "Spec Ops: The Line" game
as well as the black squares.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=97338

Signed-off-by: Samuel Pitoiset <samuel.pitoiset at gmail.com>
---

Nouveau also computes the absolute value before emitting RSQ.

 src/gallium/drivers/radeonsi/si_shader_tgsi_alu.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/src/gallium/drivers/radeonsi/si_shader_tgsi_alu.c b/src/gallium/drivers/radeonsi/si_shader_tgsi_alu.c
index 1966752cc0..ec6d6b0534 100644
--- a/src/gallium/drivers/radeonsi/si_shader_tgsi_alu.c
+++ b/src/gallium/drivers/radeonsi/si_shader_tgsi_alu.c
@@ -754,9 +754,12 @@ static void emit_rsq(const struct lp_build_tgsi_action *action,
 		     struct lp_build_tgsi_context *bld_base,
 		     struct lp_build_emit_data *emit_data)
 {
-	LLVMValueRef sqrt =
-		lp_build_emit_llvm_unary(bld_base, TGSI_OPCODE_SQRT,
-					 emit_data->args[0]);
+	LLVMValueRef abs, sqrt;
+
+	abs = lp_build_emit_llvm_unary(bld_base, TGSI_OPCODE_ABS,
+				       emit_data->args[0]);
+
+	sqrt = lp_build_emit_llvm_unary(bld_base, TGSI_OPCODE_SQRT, abs);
 
 	emit_data->output[emit_data->chan] =
 		lp_build_emit_llvm_binary(bld_base, TGSI_OPCODE_DIV,
-- 
2.11.0