[Pixman] RFC: More accurate color conversion

[Carl Worth]

Søren Sandmann <sandmann at cs.au.dk> writes:
> The correct way to convert from floating point to integers is then to
> pick the integer i such that its corresponding value is as close as
> possible to the given floating point value. In other words, the
> conversion should be done by rounding, and not by dividing the input
> range into equal-sized sections.

Hi Søren,

Thanks for bringing up this issue. It's fun to take another look at an
issue I haven't examined closely in quite some time. (And it's
heartening to see that I documented my thoughts fairly carefully at the
time.)

When considering these conversions from integers to floating-point and
back I think we[*] universally agree that all integer values must round
trip through both conversions unchanged.

What you advocate here, (and what Kalle also proposes), is that the
conversion should be chosen in such a way as to also minimize the error
when round-tripping a floating-point value to an integer and back.

I can't find evidence that I considered this metric in my argument from
2006, but it is a metric that makes sense to minimize. So that much
seems sound.

There's one part of your position that I'm unclear on. You say:

> But the output integers have inherent values, where 0x0000 corresponds
> to 0.0, 0xffff corresponds to 1.0 and 0x0001 corresponds to
> 1/65535.0. That is not an arbitrary convention -- when these integers
> are stored in framebuffers or .PNG files they are (at least in
> principle) the assumption is that they correspond to these values. The
> reason for choosing the sample points in this way, is that it is
> important to be able to represent the particular values 0.0 and 1.0
> exactly.

I agree that 0x0000 must correspond to 0.0 and that 0xffff must
correspond to 1.0.

But then you simply assert that the conversion from integer to
floating-point must be f(i) = i/65535.0. What's the justification for
this?

Other functions could be chosen for the integer->floating-point
direction to minimize the round-trip error for floating-point values
while also not violating the round-trip equivalence of integer values.

For example, Kalle proposes other functions than i/65535.0.

Is the argument that this conversion is so simple that it's safe to
assume that that's what the user had in mind when storing the integer
value into the PNG file?

I'm not necessarily arguing against the change here. I'm just pointing
out that the explanation for the change seems to depend on an implicit
assumption without much justification.

Meanwhile, I am curious what brought about your interest in changing
this function. What actual problem are you trying to solve here?

-Carl

[*] Here I'm referring at least to the following which I reviewed:

  Your message to cairo@ here
  My message to cairo@ in 2006
  Owen's web page which I referenced in my 2006 post
  The PDF by Kalle Rutanen referenced elsewhere in this thread
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/pixman/attachments/20131007/49fd8957/attachment.pgp>