[Mesa-dev] [RFC PATCH 1/2] util/f32_to_uf11: make use of optional value range

[Karol Herbst]

In the CTS test 'KHR-GL45.copy_image.functional' we ended up doing a sw format
conversion from R11F_G11F_B10F_EXT to RGBA back to R11F_G11F_B10F_EXT within
glGetTexImage calls.

By total misfortune we also got f11 values of 0x4, which got converted to a
0x36800000 f32 value in uf11_to_f32. But the result of f32_to_uf11(0x36800000)
was 0x0 which led to the above mentioned test to fail due to invalid data read
by glGetTexImage.

Adjusting f32_to_uf11 to make use of the optional value range for
	E == 0 and M != 0
so that we have more 'x' for which is the following is true
	f32_to_uf11(uf11_to_f32(x)) == x

Fixes a few subtests for KHR-GL45.copy_image.functional on Nouveau and i965.

Signed-off-by: Karol Herbst <karolherbst at gmail.com>
---
 src/util/format_r11g11b10f.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/src/util/format_r11g11b10f.h b/src/util/format_r11g11b10f.h
index ec5abf9611..5493c10403 100644
--- a/src/util/format_r11g11b10f.h
+++ b/src/util/format_r11g11b10f.h
@@ -93,6 +93,16 @@ static inline uint32_t f32_to_uf11(float val)
       exponent += UF11_EXPONENT_BIAS;
       mantissa >>= UF11_MANTISSA_SHIFT;
       uf11 = exponent << UF11_EXPONENT_SHIFT | mantissa;
+   } else if (exponent > -21) {
+      /* for exponents (-21, -15] we can be a bit smarter than returning 0
+       *
+       * From the GL_EXT_packed_float spec:
+       *      "2^-14 * (M / 64),         if E == 0 and M != 0,"
+       *
+       * based on the exponent and mantissa we can calculate M for E == 0 for
+       * the given value.
+       */
+      uf11 = (1 << (exponent + 20)) + (mantissa >> (3 - exponent));
    }
 
    return uf11;
-- 
2.14.1