[PATCH weston 00/10] Implement screenshot-based testing with the headless renderer

[Daniel Stone]

Hi Bryce,

On 7 May 2015 at 01:44, Bryce Harrington <bryce at osg.samsung.com> wrote:
> This series adds support for implementing test cases that can check
> rendering output without needing connection to a physical output.  It
> utilizes the pixman renderer in the headless backend to generate
> screenshots at the request of the test client.
>
> The test creates a shm buffer and passes it to the server via the
> Weston-Test protocol ('capture_screenshot').  The server then renders
> the display contents into the buffer and returns it as a response
> ('capture_screenshot_done').
>
> The test then loads a corresponding reference PNG from disk as a cairo
> surface, and then compares it with the captured screenshot.  Note that
> the screenshot includes the current time in the desktop clock, which
> will of course be different in the captured screenshot from the
> reference image.  So this checks a small clipped out section of each of
> the two images to verify congruence.
>
> By default, Wayland fades in the display and will show a patterned
> background.  The former feature causes a black or nearly-black image to
> be captured (the darkness of which may vary from system to system), and
> the latter merely adds noise in our comparison, so both features are
> disabled via a test-specific configuration .ini file.

Modulo the comments I had earlier (and loads of C++ comments in
09/10), these do look good to me.

Thanks a lot for doing them - they're very useful, and I'd like to see
them get into 1.8 if possible.

They're certainly not perfect, and I think show up a lot of the
limitations of our current test 'infrastructure', but IMO that should
in no way block these tests. On the contrary, it gives us more data to
build our test framework on: it will make the conversion a little more
painful, but I think be a good guide for its development.

Cheers,
Daniel