[Mesa-dev] [PATCH] swrast: fix readpix clamping

[Marek Olšák]

Broken with e5c6a92a12b5cd7db205d72039f58d302b0be9d5. (ARB_color_buffer_float)

Clamping should occur if type != float, otherwise the MSBs of the resulting
pixels are killed off. For example, reading back LUMIMANCE = R+G+B can be
greater than 0xff, but the result is naturally masked by 0xff
for UNSIGNED_BYTE, leading to bogus results.

The following bug report seems to want clamping to occur if type == half_float
too. Not sure what's correct.

Bug: [bisected pineview] oglc case pxconv-read failed
https://bugs.freedesktop.org/show_bug.cgi?id=35852
---
 src/mesa/swrast/s_readpix.c |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/mesa/swrast/s_readpix.c b/src/mesa/swrast/s_readpix.c
index 5604c2e..9e0f0d5 100644
--- a/src/mesa/swrast/s_readpix.c
+++ b/src/mesa/swrast/s_readpix.c
@@ -315,12 +315,12 @@ read_rgba_pixels( struct gl_context *ctx,
    if (!rb)
       return;
 
-   if (type == GL_FLOAT && ((ctx->Color.ClampReadColor == GL_TRUE) ||
-                            (ctx->Color.ClampReadColor == GL_FIXED_ONLY_ARB &&
-                             rb->DataType != GL_FLOAT)))
+   if ((ctx->Color._ClampReadColor == GL_TRUE || type != GL_FLOAT) &&
+       !_mesa_is_integer_format(format)) {
       transferOps |= IMAGE_CLAMP_BIT;
+   }
 
-   /* Try optimized path first */
+   /* Try the optimized path first. */
    if (fast_read_rgba_pixels(ctx, x, y, width, height,
                              format, type, pixels, packing, transferOps)) {
       return; /* done! */
-- 
1.7.1