conversion from double to fixed and back fails with certain values

[Jason Ekstrand]

Micah,
This is because wl_fixed is a fixed-point format.  In particular, it is
24.8 fixed point meaning that the top  24 bits represent the integer part.
This means that wl_fixed effectively stores n if the number is written as
the (possibly improper) fraction n/256.  In your example, 3568.005 =
913409.28 / 256.  Because wl_fixed only storres the integer n, this gets
truncated to 913409/256.  When it gets converted back, this obviously does
not give you the same floating-point value.
I hope that makes sense,
--Jason Ekstrand



On Thu, Sep 26, 2013 at 3:40 PM, Micah Nordland <mpnordland at gmail.com>wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> The value 3568.005 is not correctly converted back from fixed.
> The following patch to tests/fixed-test.c demonstrates. Is this
> expected behavior?
>
> diff --git a/tests/fixed-test.c b/tests/fixed-test.c
> index 739a3b1..89ec188 100644
> - --- a/tests/fixed-test.c
> +++ b/tests/fixed-test.c
> @@ -65,6 +65,11 @@ TEST(fixed_double_conversions)
>         d = wl_fixed_to_double(f);
>         fprintf(stderr, "fixed %x to double %lf\n", f, d);
>         assert(d == f / 256);
> +
> +    d = 3568.005;
> +    f = wl_fixed_from_double(d);
> +    fprintf(stderr, "fixed %x to double %lf\n", f, d);
> +    assert(d == wl_fixed_to_double(f));
>  }
>
>  TEST(fixed_int_conversions)
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.21 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJSRJuxAAoJEPBh3Rt3H/83HkYH/2w1lrzcCNihoPp3zOrLmxao
> 8CdSvdIHvgAWvC6mnqJx9bL7W6bqI0LrZHRgCJqKFQcvnyQq9k5k+s2Kh1C87SbQ
> w8KQ3B4bkRl+m4oM4mxmbeQAlHKi0+mcyu8w6bUG/8OZRvGNvfjqUW2p62MHTKUr
> W9Uw9cq5A30I11LV13G5enFP1JIzVJqjoVykOJd4sSRX5vdPzr+n14ThSFQ/vu8D
> MjpZgHvKFZ5LPkCWb0MulQc7LXAqpabJTlMlDZ56HcbqevCZ6sg1AEd6DMdKqvXi
> lm17vVEOOJL6SLiNruTMEyi1TFi0+B06h972VH8x9RKg1LnhLtHsfu9JjiRdLiQ=
> =96Vx
> -----END PGP SIGNATURE-----
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20130926/ae5caa65/attachment.html>