Mesa (master): mesa/vbo: Fix scaling issue in 10-bit signed normalized packing.
Kenneth Graunke
kwg at kemper.freedesktop.org
Thu Nov 22 05:15:25 UTC 2012
Module: Mesa
Branch: master
Commit: e9967aba61a7cc62ce13ab691d14bb5a26a86896
URL: http://cgit.freedesktop.org/mesa/mesa/commit/?id=e9967aba61a7cc62ce13ab691d14bb5a26a86896
Author: Kenneth Graunke <kenneth at whitecape.org>
Date: Fri Oct 12 11:17:39 2012 -0700
mesa/vbo: Fix scaling issue in 10-bit signed normalized packing.
For the 10-bit components, the divisor was incorrect. A 10-bit signed
integer can represent -2^9 through 2^9 - 1, which leads to the following
ranges:
(float)value.x -> [ -512, 511]
2.0F * (float)value.x -> [-1024, 1022]
2.0F * (float)value.x + 1.0F -> [-1023, 1023]
So dividing by 511 would incorrectly scale it to approximately:
[-2.001956947, 2.001956947]. To correctly scale to [-1.0, 1.0], we need
to divide by 1023.
This correctly implements the desktop GL rules. ES 3.0 has different
rules, but those will be implemented in a separate patch.
Signed-off-by: Kenneth Graunke <kenneth at whitecape.org>
Tested-by: Marek Olšák <maraeo at gmail.com>
---
src/mesa/vbo/vbo_attrib_tmp.h | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/src/mesa/vbo/vbo_attrib_tmp.h b/src/mesa/vbo/vbo_attrib_tmp.h
index 6bc53ba..9a4df69 100644
--- a/src/mesa/vbo/vbo_attrib_tmp.h
+++ b/src/mesa/vbo/vbo_attrib_tmp.h
@@ -117,7 +117,7 @@ static inline float conv_i10_to_norm_float(int i10)
{
struct attr_bits_10 val;
val.x = i10;
- return (2.0F * (float)val.x + 1.0F) * (1.0F / 511.0F);
+ return (2.0F * (float)val.x + 1.0F) * (1.0F / 1023.0F);
}
static inline float conv_i2_to_norm_float(int i2)
More information about the mesa-commit
mailing list