[Openicc] ICC meta Tag for Monitor Profiles 1.0 - DRAFT 5

[Richard Hughes]

On 15 January 2012 15:02, Kai-Uwe Behrmann <ku.b at gmx.de> wrote:
>>> * EDID_red_x:
>>> * EDID_red_y:
>> What floating point precision? For instance, if GCM writes "0.8789123"
>> and Oyranos writes "0.87891229" would that match?
> Applications should write to the EDID delivered precission. A link to the
> EDID spec is in the DRAFT. "2.2" and "2.2000000" are numerically the same.

Okay, but there is no numerical precision specified for the primaries.
A client has to do bit manipulation, followed by repeat multiplication
of a power of two to another value. It's perfectly conceivable that
the same code on i386 and ppc64 is going to produce slightly different
results in this case. Comparing floating points for equality like this
is not a good idea at all. The spec should really have some kind of
guideline for a good enough match, i.e. a delta of =< 0.0001

Richard.