Mesa (main): util: Keep quiet NaNs quiet when converting to half float.

[GitLab Mirror]

Module: Mesa
Branch: main
Commit: 27e33d5c963f13a47c4b9276378db913d8ecb189
URL:    http://cgit.freedesktop.org/mesa/mesa/commit/?id=27e33d5c963f13a47c4b9276378db913d8ecb189

Author: Chia-I Wu <olvaffe at gmail.com>
Date:   Thu Apr 28 14:25:36 2022 -0700

util: Keep quiet NaNs quiet when converting to half float.

We don't want to be throwing exceptions and changing float values later by
emitting a signaling binary16 nan.

If we don't do this, then when we convert back to f32 in NIR constant
expression evaluation, the signaling NaN can end up giving NaN for
fmax(NaN, 0.0), instead of 0.0.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5933
Cc: mesa-stable
Reviewed-by: Emma Anholt <emma at anholt.net>
Reviewed-by: Jason Ekstrand <jason.ekstrand at collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16233>

---

 src/util/half_float.c | 8 ++++++--
 src/util/softfloat.c  | 7 ++++++-
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/src/util/half_float.c b/src/util/half_float.c
index 05aeac14abb..606e4b9522e 100644
--- a/src/util/half_float.c
+++ b/src/util/half_float.c
@@ -83,8 +83,12 @@ _mesa_float_to_half_slow(float val)
       e = 31;
    }
    else if ((flt_e == 0xff) && (flt_m != 0)) {
-      /* NaN */
-      m = 1;
+      /* Retain the top bits of a NaN to make sure that the quiet/signaling
+       * status stays the same.
+       */
+      m = flt_m >> 13;
+      if (!m)
+         m = 1;
       e = 31;
    }
    else {
diff --git a/src/util/softfloat.c b/src/util/softfloat.c
index cc386aa46fb..2ff835f7452 100644
--- a/src/util/softfloat.c
+++ b/src/util/softfloat.c
@@ -1452,7 +1452,12 @@ _mesa_float_to_half_rtz_slow(float val)
         if (flt_m != 0) {
             /* 'val' is a NaN, return NaN */
             e = 0x1f;
-            m = 0x1;
+            /* Retain the top bits of a NaN to make sure that the quiet/signaling
+            * status stays the same.
+            */
+            m = flt_m >> 13;
+            if (!m)
+               m = 1;
             return (s << 15) + (e << 10) + m;
         }