[Piglit] [PATCH] CL: Fix check of ULP when probing float/double results
Jan Vesely
jan.vesely at rutgers.edu
Sun Jan 3 15:17:02 PST 2016
Hi Aaron,
On Tue, 2015-12-29 at 12:01 -0600, Aaron Watry wrote:
> Hi Jan/Tom,
>
> Sorry to resurrect an ancient thread, but I was poking at the piglit
> CL ULP
> issue last night, and thought I'd try to get us closer to a solution
> after
> dropping the matter for too long.
>
> I've modified your test program with what I was thinking of trying,
> and I
> wouldn't mind your feedback.
thanks for picking it up.
>
> I realize that we still have issues with discrepancies between
> python-generated expected results based on cpu/rounding mode and the
> , but
> I'd at least like for us to be able to nail down the C portion before
> we
> start redefining all of our expected test results. In the long term,
> we
> probably want to hand-select our inputs/outputs instead of trusting
> python,
> but I don't necessarily think that we should block fixing our ULP
> calculations on getting that done.
I think the python issue is separate, although related. We should
assume that whatever provides the reference values is reliable.
>
> For now, I'm still ignoring the half-ULP possibility, and just
> generating a
> minimum/maximum allowed value based on running nextafterf(expected,
> POS/NEG
> INFINITY) in a loop for ULP iterations. Does that sound like a
> tenable
> solution?
I originally considered the same approach, and it has probably the best
effort/correctness ratio. I think we should take something similar
without waiting for a perfect solution. for more details read below.
Here are couple of problems I see:
a) the approach assumes that the nextafter distances are always the
same, which is not always the case.
b) the specs say "The reference value used to compute the ULP value of
an arithmetic operation is the infinitely precise result." which in my
understanding means that we compare GPU (FP representable) result
against infinitely precise mathematical value.
the provided value (python reference - PRef) is representable and
rounded. thus the ULP reference is anywhere in <PRef - Er_b, PRef +
Er_a>, where Er_{a,b} represents rounding interval values for
above/below, and ULP_{a,b} the corresponding ULP values.
The nextafter approach really checks against N * ULP_a + Er_b, or N *
ULP_b + Er_a.
Even if ULP_a == ULP_b (in all steps) the value PRef - N * ULP is
correct if the infinitely precise result is in <PRef - Er_b, PRef> and
incorrect of it is in <PRef, PRef + Er_a>, and we have no way to
distinguish these cases.
Thus, if you don't use strict inequalities you already test (N + 0.5)
ULP (since Er_{a,b} <= 0.5 ULP, for round to nearest modes).
Given that we don't have any information about the infinitely precise
result we don't even know what ULP value to use for the test if PRef
happens to be a boundary value.
example:
2^24 is a boundary value for SP float; the neighbouring representable
values are (in order):
..., 2^24 -2, 2^24 -1, 2^24, 2^24 +2, 2^24 +4, ...
Thus if python provides us with PRef == 2^24 using RTNE(*) rounding,
the correct value (Ref) can be anything in <2^24 - 0.5, 2^25>.
If the required accuracy is 2 ULP and Ref == 2^24 - 0.3 (ULP = 1.0),
then the correct test is: GPU_result >= 2^24 - 2 && GPU_result < 2^24 +
2. ( |GPU_Result - Ref| <= 2.0)
if Ref == 2^24 + 0.5 (ULP = 2.0) then the correct test is: GPU_result >
2^24 - 4 && GPU_result <= 2^24 + 4. ( |GPU_Result - Ref| <= 4.0)
Note the strictness of inequalities in both cases. changing precision
to 2.5 ULP would just change the inequalities to non-strict
regards,
Jan
(*) even is a bit misnomer since all SP floats >= 2.24 are even, I
assume it means that mantissa is even
>
> --Aaron
>
> On Sun, Jun 14, 2015 at 4:19 PM, Jan Vesely <jan.vesely at rutgers.edu>
> wrote:
>
> > On Sat, 2015-06-13 at 21:22 -0500, Aaron Watry wrote:
> > > Meh, this still feels broken. Give me a bit longer.
> >
> > and it is :). I don't think this can work based on abs(expected -
> > real),
> > since ULP depends on the magnitude of the numbers. This information
> > is
> > lost after subtraction.
> >
> > I had an idea some time back to implement this using nextafterf in
> > both
> > directions and checking whether the result falls in that interval.
> > However, there is still a problem. Some of the expected values are
> > already rounded (I'm not sure what rounding mode is used by python
> > by
> > default), but unless it always rounds in one direction, we'll still
> > get
> > slight differences based on whether the actual value was rounded up
> > or
> > down.
> >
> > I have attached a small test program that shows the deficiencies of
> > fabsf based approaches.
> >
> > regards,
> > Jan
> >
> > >
> > > --Aaron
> > >
> > > On Sat, Jun 13, 2015 at 2:28 PM, Aaron Watry <awatry at gmail.com>
> > > wrote:
> > >
> > > > We need to actually check against the float value from the
> > > > union,
> > > > instead of just doing (diff > ulp), which seems to cast the
> > > > diff to
> > > > an int before checking against ulp.
> > > >
> > > > Signed-off-by: Aaron Watry <awatry at gmail.com>
> > > > CC: Tom Stellard <thomas.stellard at amd.com>
> > > > CC: Jan Vesely <jan.vesely at rutgers.edu>
> > > > ---
> > > > tests/util/piglit-util-cl.c | 28 ++++++++++++++--------------
> > > > 1 file changed, 14 insertions(+), 14 deletions(-)
> > > >
> > > > diff --git a/tests/util/piglit-util-cl.c b/tests/util/piglit-
> > > > util-cl.c
> > > > index 47e0c7a..6cdd718 100644
> > > > --- a/tests/util/piglit-util-cl.c
> > > > +++ b/tests/util/piglit-util-cl.c
> > > > @@ -80,7 +80,7 @@ piglit_cl_probe_floating(float value, float
> > > > expect,
> > > > uint32_t ulp)
> > > >
> > > > diff = fabsf(value - expect);
> > > >
> > > > - if(diff > ulp || isnan(value)) {
> > > > + if (diff > t.f || isnan(value)) {
> > > > printf("Expecting %f (0x%x) with tolerance %f
> > > > (%u
> > ulps),
> > > > but got %f (0x%x)\n",
> > > > e.f, e.u, t.f, t.u, v.f, v.u);
> > > > return false;
> > > > @@ -108,7 +108,7 @@ piglit_cl_probe_double(double value, double
> > > > expect,
> > > > uint64_t ulp)
> > > >
> > > > diff = fabsl(value - expect);
> > > >
> > > > - if(diff > ulp || isnan(value)) {
> > > > + if (diff > t.f || isnan(value)) {
> > > > printf("Expecting %f (0x%lx) with tolerance %f
> > > > (%lu
> > ulps),
> > > > but got %f (0x%lx)\n",
> > > > e.f, e.u, t.f, t.u, v.f, v.u);
> > > > return false;
> > > > @@ -162,7 +162,7 @@
> > > > piglit_cl_get_platform_version(cl_platform_id
> > platform)
> > > > int scanf_count;
> > > > int major;
> > > > int minor;
> > > > -
> > > > +
> > > > /*
> > > > * Returned format:
> > > > *
> > > > OpenCL<space><major_version.minor_version><space><platform-
> > > > specific
> > > > information>
> > > > @@ -353,7 +353,7 @@ piglit_cl_get_info(void* fn_ptr, void* obj,
> > > > cl_uint
> > > > param)
> > > >
> > > > if(errNo == CL_SUCCESS) {
> > > > param_ptr = calloc(param_size, sizeof(char));
> > > > -
> > > > +
> > > > /* retrieve param */
> > > > if(fn_ptr == clGetPlatformInfo) {
> > > > errNo =
> > clGetPlatformInfo(*(cl_platform_id*)obj,
> > > > param,
> > > > @@ -463,7 +463,7 @@ piglit_cl_get_program_build_info(cl_program
> > program,
> > > > cl_device_id device,
> > > > .program = program,
> > > > .device = device
> > > > };
> > > > -
> > > > +
> > > > return piglit_cl_get_info(clGetProgramBuildInfo, &args,
> > > > param);
> > > > }
> > > >
> > > > @@ -479,7 +479,7 @@
> > > > piglit_cl_get_kernel_work_group_info(cl_kernel
> > kernel,
> > > > cl_device_id device,
> > > > .kernel = kernel,
> > > > .device = device
> > > > };
> > > > -
> > > > +
> > > > return piglit_cl_get_info(clGetKernelWorkGroupInfo,
> > > > &args,
> > param);
> > > > }
> > > >
> > > > @@ -620,7 +620,7 @@ piglit_cl_get_device_ids(cl_platform_id
> > platform_id,
> > > > cl_device_type device_type,
> > > >
> > piglit_cl_get_error_name(errNo));
> > > > return 0;
> > > > }
> > > > -
> > > > +
> > > > /* get device list */
> > > > if(device_ids != NULL && num_device_ids
> > > > > 0) {
> > > > *device_ids =
> > > > malloc(num_device_ids *
> > > > sizeof(cl_device_id));
> > > > @@ -761,7 +761,7 @@
> > > > piglit_cl_build_program_with_source_extended(piglit_cl_context
> > > > context,
> > > > piglit_cl_get_error_name(errNo));
> > > > return NULL;
> > > > }
> > > > -
> > > > +
> > > > errNo = clBuildProgram(program,
> > > > context->num_devices,
> > > > context->device_ids,
> > > > @@ -788,7 +788,7 @@
> > > > piglit_cl_build_program_with_source_extended(piglit_cl_context
> > > > context,
> > > > char* log =
> > > > piglit_cl_get_program_build_info(program,
> > > >
> > > > context->device_ids[i],
> > > >
> > > > CL_PROGRAM_BUILD_LOG);
> > > > -
> > > > +
> > > > printf("Build log for device %s:\n ----
> > > > ----
> > \n%s\n
> > > > -------- \n",
> > > > device_name,
> > > > log);
> > > > @@ -848,11 +848,11 @@
> > > > piglit_cl_build_program_with_binary_extended(piglit_cl_context
> > > > context,
> > > > for(i = 0; i < context->num_devices; i++) {
> > > > char* device_name =
> > > > piglit_cl_get_device_info(context->device_ids[i],
> > > >
> > > > CL_DEVICE_NAME);
> > > > -
> > > > +
> > > > printf("Error for %s: %s\n",
> > > > device_name,
> > > >
> > piglit_cl_get_error_name(binary_status[i]));
> > > > -
> > > > +
> > > > free(device_name);
> > > > }
> > > >
> > > > @@ -860,7 +860,7 @@
> > > > piglit_cl_build_program_with_binary_extended(piglit_cl_context
> > > > context,
> > > > return NULL;
> > > > }
> > > > free(binary_status);
> > > > -
> > > > +
> > > > errNo = clBuildProgram(program,
> > > > context->num_devices,
> > > > context->device_ids,
> > > > @@ -884,11 +884,11 @@
> > > > piglit_cl_build_program_with_binary_extended(piglit_cl_context
> > > > context,
> > > > char* log =
> > > > piglit_cl_get_program_build_info(program,
> > > >
> > > > context->device_ids[i],
> > > >
> > > > CL_PROGRAM_BUILD_LOG);
> > > > -
> > > > +
> > > > printf("Build log for device %s:\n ----
> > > > ----
> > \n%s\n
> > > > -------- \n",
> > > > device_name,
> > > > log);
> > > > -
> > > > +
> > > > free(device_name);
> > > > free(log);
> > > > }
> > > > --
> > > > 2.1.4
> > > >
> > > >
> >
> >
> > --
> > Jan Vesely <jan.vesely at rutgers.edu>
> >
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20160104/7b9c3fb6/attachment.sig>
More information about the Piglit
mailing list