Mesa (master): util: return 0 for NaNs in float_to_ubyte

[GitLab Mirror]

Module: Mesa
Branch: master
Commit: c72f91debab68d2c0ac7f71807a451b696859afd
URL:    http://cgit.freedesktop.org/mesa/mesa/commit/?id=c72f91debab68d2c0ac7f71807a451b696859afd

Author: Roland Scheidegger <sroland at vmware.com>
Date:   Fri Aug  3 05:34:07 2018 +0200

util: return 0 for NaNs in float_to_ubyte

d3d10 requires NaNs to get converted to 0 for float->unorm conversions
(and float->int etc.). GL spec probably doesn't care in general, but it
would make sense to have reasonable behavior in any case imho - the
old code was converting negative NaNs to 0, and positive NaNs to 255.
(Note that using float comparison isn't actually all that much more
effort in any case, at least with sse2 it's just float comparison
(ucommiss) instead of int one - I converted the second comparison
to float too simply because it saves the probably somewhat expensive
transfer of the float from simd to int domain (with sse2 via stack),
so the generated code actually has 2 less instructions, although float
comparisons are more expensive than int ones.)

Reviewed-by: Brian Paul <brianp at vmware.com>

---

 src/gallium/auxiliary/util/u_math.h | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/src/gallium/auxiliary/util/u_math.h b/src/gallium/auxiliary/util/u_math.h
index 79869a119a..712305c98a 100644
--- a/src/gallium/auxiliary/util/u_math.h
+++ b/src/gallium/auxiliary/util/u_math.h
@@ -360,7 +360,6 @@ uif(uint32_t ui)
 
 /**
  * Convert ubyte to float in [0, 1].
- * XXX a 256-entry lookup table would be slightly faster.
  */
 static inline float
 ubyte_to_float(ubyte ub)
@@ -375,16 +374,16 @@ ubyte_to_float(ubyte ub)
 static inline ubyte
 float_to_ubyte(float f)
 {
-   union fi tmp;
-
-   tmp.f = f;
-   if (tmp.i < 0) {
+   /* return 0 for NaN too */
+   if (!(f > 0.0f)) {
       return (ubyte) 0;
    }
-   else if (tmp.i >= 0x3f800000 /* 1.0f */) {
+   else if (f >= 1.0f) {
       return (ubyte) 255;
    }
    else {
+      union fi tmp;
+      tmp.f = f;
       tmp.f = tmp.f * (255.0f/256.0f) + 32768.0f;
       return (ubyte) tmp.i;
    }