[igt-dev] [PATCH i-g-t v3 18/21] lib/igt_frame: Add a checkerboard frame comparison method

[Maxime Ripard]

On Fri, Jan 11, 2019 at 10:05:29AM +0100, Paul Kocialkowski wrote:
> This introduces a new frame comparison method that was designed for
> patterns that follow a checkerboard style. These patterns are made of
> consecutive rectangular shapes with alternating solid colors. They are
> currently used for some Chamelium-based tests.
> 
> The method is particularly adapted for cases where the edges of the
> shapes might be blurred (e.g. due to scaling), which makes it impossible
> to use pixel-perfect or CRC-based comparisons to decide whether the
> captured frame matches the reference.
> 
> Overall, this test will first detect the edges of the pattern and later
> exclude them from comparison. Colors are compared between the reference
> and capture with a low threshold for error. A percentage of the faulty
> pixels is calculated and the captured frame is considered invalid if
> more than one percent of the pixels are erroneous.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski at bootlin.com>
> ---
>  lib/igt_frame.c | 121 ++++++++++++++++++++++++++++++++++++++++++++++++
>  lib/igt_frame.h |   2 +
>  2 files changed, 123 insertions(+)
> 
> diff --git a/lib/igt_frame.c b/lib/igt_frame.c
> index 6984c02e9912..e0215660ea6e 100644
> --- a/lib/igt_frame.c
> +++ b/lib/igt_frame.c
> @@ -267,3 +267,124 @@ complete:
>  
>  	return match;
>  }
> +
> +#define XR24_COLOR_VALUE(data, stride, x, y, c) \
> +	*((uint8_t *)(data) + (y) * (stride) + 4 * (x) + (c))
> +
> +/**
> + * igt_check_checkerboard_frame_match:
> + * @reference: The reference cairo surface
> + * @capture: The captured cairo surface
> + *
> + * Checks that the reference frame matches the captured frame using a
> + * method designed for checkerboard patterns. These patterns are made of
> + * consecutive rectangular shapes with alternating solid colors.
> + *
> + * The intent of this method is to cover cases where the captured result is
> + * pixel-perfect due to features such as scaling or YUV conversion and

is *not* pixel perfect?

> + * subsampling. Such effects are mostly noticeable on the edges of the
> + * patterns, so they are detected and excluded from the comparison.
> + *
> + * Returns: a boolean indicating whether the frames match
> + */
> +bool igt_check_checkerboard_frame_match(cairo_surface_t *reference,
> +					cairo_surface_t *capture)
> +{
> +	unsigned int width, height, ref_stride, cap_stride;
> +	void *ref_data, *cap_data;
> +	unsigned char *edges_map;
> +	unsigned int x, y, c;
> +	unsigned int errors = 0, pixels = 0;
> +	unsigned int edge_threshold = 100;
> +	unsigned int color_error_threshold = 24;
> +	double error_rate_threshold = 0.01;
> +	double error_rate;
> +	unsigned int span = 2;
> +	bool match = false;
> +
> +	width = cairo_image_surface_get_width(reference);
> +	height = cairo_image_surface_get_height(reference);
> +
> +	ref_stride = cairo_image_surface_get_stride(reference);
> +	ref_data = cairo_image_surface_get_data(reference);
> +	igt_assert(ref_data);
> +
> +	cap_stride = cairo_image_surface_get_stride(capture);
> +	cap_data = cairo_image_surface_get_data(capture);
> +	igt_assert(cap_data);
> +
> +	edges_map = calloc(1, width * height);
> +
> +	/* First pass to detect the pattern edges. */
> +	for (y = 0; y < height; y++) {
> +		if (y < span || y > (height - span - 1))
> +			continue;
> +
> +		for (x = 0; x < width; x++) {
> +			unsigned int xdiff = 0, ydiff = 0;
> +
> +			if (x < span || x > (width - span - 1))
> +				continue;
> +
> +			for (c = 0; c < 3; c++) {
> +				xdiff += abs(XR24_COLOR_VALUE(ref_data, ref_stride, x + span, y, c) -
> +					     XR24_COLOR_VALUE(ref_data, ref_stride, x - span, y, c));
> +				ydiff += abs(XR24_COLOR_VALUE(ref_data, ref_stride, x, y + span, c) -
> +					     XR24_COLOR_VALUE(ref_data, ref_stride, x, y - span, c));
> +			}
> +
> +			edges_map[y * width + x] = (xdiff > edge_threshold ||
> +						    ydiff > edge_threshold);
> +		}
> +	}
> +
> +	/* Second pass to detect errors. */
> +	for (y = 0; y < height; y++) {
> +		for (x = 0; x < width; x++) {
> +			bool error = false;
> +
> +			if (edges_map[y * width + x])
> +				continue;
> +
> +			for (c = 0; c < 3; c++) {
> +				unsigned int diff;
> +
> +				/* Compare the reference and capture values. */
> +				diff = abs(XR24_COLOR_VALUE(ref_data, ref_stride, x, y, c) -
> +					   XR24_COLOR_VALUE(cap_data, cap_stride, x, y, c));
> +
> +				if (diff > color_error_threshold)
> +					error = true;
> +			}
> +
> +			/* Allow error if coming on or off an edge (on x). */
> +			if (error && x >= span && x <= (width - span - 1) &&
> +			    edges_map[y * width + (x - span)] !=
> +			    edges_map[y * width + (x + span)])
> +				continue;
> +
> +			/* Allow error if coming on or off an edge (on y). */
> +			if (error && y >= span && y <= (height - span - 1) &&
> +			    edges_map[(y - span) * width + x] !=
> +			    edges_map[(y + span) * width + x] && error)
> +				continue;
> +
> +			if (error)
> +				errors++;
> +
> +			pixels++;
> +		}
> +	}
> +
> +	free(edges_map);
> +
> +	error_rate = (double) errors / pixels;
> +
> +	if (error_rate < error_rate_threshold)
> +		match = true;
> +
> +	igt_debug("Checkerboard pattern %s with error rate %f %%\n",
> +		  match ? "matched" : "not matched", error_rate * 100);
> +
> +	return match;
> +}
> diff --git a/lib/igt_frame.h b/lib/igt_frame.h
> index 11f96cbea203..f44f57d7ce73 100644
> --- a/lib/igt_frame.h
> +++ b/lib/igt_frame.h
> @@ -38,5 +38,7 @@ void igt_write_compared_frames_to_png(cairo_surface_t *reference,
>  				      const char *capture_suffix);
>  bool igt_check_analog_frame_match(cairo_surface_t *reference,
>  				  cairo_surface_t *capture);
> +bool igt_check_checkerboard_frame_match(cairo_surface_t *reference,
> +					cairo_surface_t *capture);

Looks good otherwise. I guess it's missing some comments about the
algorithm being used though.

Maxmie

-- 
Maxime Ripard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/igt-dev/attachments/20190111/d82d18ef/attachment.sig>