[Mesa-dev] SNORM conversion equation woes

[Ian Romanick]

On 07/26/2013 02:20 PM, Kenneth Graunke wrote:
> Hello all,
>
> I've been looking at https://bugs.freedesktop.org/show_bug.cgi?id=59150
> and could use your opinion.
>
> OpenGL defines two equations for converting from SNORM to floating point
> data.  These are:
>
> f = (2c + 1)/(2^b - 1)                (equation 2.2)
> f = max{c/2^(b-1) - 1), -1.0}         (equation 2.3)
>
> Equation 2.2 cannot represent 0.0f, while equation 2.3 can.
> Equation 2.3 has two representations for -1.
>
> The spec language for which to use is a bit wishy-washy...GL 3.2:
> "In general, [equation 2.2] is used for signed normalized fixed-point
>   parameters in GL commands, such as vertex attribute values."
> "In general, [equation 2.3] is used for signed normalized fixed-point
>   texture or floating point values."
>
> ARB_vertex_type_2_10_10_10_rev clearly specifies equation 2.2.
>
> *However*, OpenGL 4.2 and ES 3.0 both remove equation 2.2, instead
> saying to use equation 2.3 in all cases.
>
> Intel hardware always uses equation 2.3, which provides a dilemma:
> should we just use the native hardware rules, which are fast, or do
> equation 2.2 via shader workarounds?
>
> The maximum error/difference between the two equations is:
>
> 16-bit -> 0.00003
> 10-bit -> 0.00195
>   8-bit -> 0.00784
>   2-bit -> 0.66666
>
> (Data and a script from Keith are available from:)
>   http://people.freedesktop.org/~kwg/snorm/
>
> Furthermore, since GL 4.2 is supposed to be backwards compatible,
> implementations can silently promote contexts to 4.2, likely giving them
> the new SNORM rules.  Which means applications can't really rely on the
> exact equation used...
>
> Always using 2.3 seems preferable since it can actually represent 0.
>
> I am tempted to rip out the code in vbo_attrib_tmp.h that implements
> equation 2.2 and always use 2.3.  That would also require changing
> Piglit's draw-vertices-2101010 test to accept either equation.
>
> What do you all think?

Okay... In OpenGL 4.2 we changed to use one set of rules for 3 reasons:

1. OpenGL ES 3.0 only wanted one set of rules, and we wanted to avoid 
divergence between the specs.

2. DX10 had already changed to use equation 2.3.

3. Some shipping hardware only supported equation 2.3... even for 
pre-4.2 contexts.

Item #3 was the last nail in the coffin, so to speak.  It also means 
that applications can't depend on equation 2.2.  More than one shipping 
implementation of GL_ARB_vertex_type_2_10_10_10_rev uses equation 2.3 
even though the spec gives equation 2.2.  As a result, applications 
can't depend on equation 2.2. :(

Here are my inferences based on that:

  - Applications that are porting from DX to GL will most likely want 
equation 2.3 regardless of the GL version advertised.

  - New OpenGL and OpenGL ES "native" applications will most likely want 
equation 2.3.

My suggestions for Mesa:

1. OpenGL ES 3.0 contexts must use equation 2.3.

2. OpenGL 4.2 contexts must use equation 2.3.

3. The piglit test should be extended to accept implementations that 
implement either equation 2.2 or equation 2.3.

4. Mesa should default to equation 2.3.  Hardware that can do equation 
2.2 should have a driconf option for applications that really depend on 
equation 2.2.  I honestly doubt that there are (or ever will be) any.

I don't think there's much room for argument about the first three, but 
#4 isn't quite so clear cut.  Does this sound okay to folks?

> Thanks for your time!
>    --Ken