[PATCH weston v3 00/15] Implement screenshot-based testing with the headless renderer

[Bryce Harrington]

On Thu, May 21, 2015 at 04:44:41PM -0500, Derek Foreman wrote:
> With the tiny stride change mentioned on IRC, which, for those that
> weren't present, was basically little more than:
> if (stride != bpp * reference->width) FAIL
> in load_surface_from_png()
> 
> for the series:
> Reviewed-By: Derek Foreman <derekf at osg.samsung.com>

Thanks, landed (including the additional stride change):
   f556ebe..0ccf8e2  master -> master

> On 21/05/15 03:39 AM, Bryce Harrington wrote:
> > Thought I'd try one last run at the football...
> > 
> > This series adds support for implementing test cases that can check
> > rendering output without needing connection to a physical output.  It
> > utilizes the pixman renderer in the headless backend to generate
> > screenshots at the request of the test client.
> > 
> > The test creates a shm buffer and passes it to the server via the
> > Weston-Test protocol ('capture_screenshot').  The server then renders
> > the display contents into the buffer and returns it as a response
> > ('capture_screenshot_done').
> > 
> > The test then loads a corresponding reference PNG from disk using Cairo,
> > and then compares it with the captured screenshot.  Note that the
> > screenshot includes the current time in the desktop clock, which will of
> > course be different in the captured screenshot from the reference image.
> > So this checks a small clipped out section of each of the two images to
> > verify congruence.
> > 
> > By default, Wayland fades in the display and will show a patterned
> > background.  The former feature causes a black or nearly-black image to
> > be captured (the darkness of which may vary from system to system), and
> > the latter merely adds noise in our comparison, so both features are
> > disabled via a test-specific configuration .ini file.
> > 
> > [Update v2]
> >  * Refactor cairo out of the test client backend code entirely
> >    by utilizing the weston test surface structure to carry the
> >    specific data that the check_ routines need.
> >  * Drop refactoring redundant pixel copying code to shared
> >  * Take stride and bpp into account in check functions
> >  * In test case, set the bpp for the screenshot
> >  * Fix screenshot protocol description
> >  * Convert C++ style comments to C
> >  * Don't leak screenshot object
> >  * s/ec/compositor/
> >  * Instead of spinning on client_roundtrip, check for client's wl_display
> >  * Drop unneeded wl_surface in test case
> >  * Restore missing weston_log in idle_launch_client()
> >  * Drop unused test object from capture_screenshot
> >  * check_ function variable cleanup
> >  * Prefer tracking stride rather than bpp
> >  * Cleanup character calculation in check_ routines
> >  * Drop unnecessary data copying before write to disk
> >  * Split up test case code into several functions
> >  * Use screenshot_output_filename() so output files respect the
> >    WESTON_TEST_OUTPUT_PATH env var
> > 
> > [Update v3]
> >  * Stripped out support for handling dissimilar strides.  This probably
> >    will never come up in practice; if it does it can be handled then.
> >  * Improve code style in a few places.
> > 
> > 
> > Bryce Harrington (15):
> >   tests: Add error handling for system calls
> >   tests: Add client helper routines for output and reference filenames
> >   tests: Add an xmalloc helper function
> >   tests: Add surface checks
> >   tests: Support --config to enable tests to override config defaults
> >   protocol: Add test screenshot capability
> >   tests: Add screenshot recording capability to weston-test
> >   tests: Handle screenshot done event in weston-test
> >   tests: Add internal test for the weston test screenshot capability
> >   tests: Add write_surface_as_png() helper
> >   tests: Add create_screenshot_surface()
> >   tests: Add load_surface_from_png()
> >   tests: Add check_surfaces_geometry()
> >   tests: Add capture_screenshot_of_output()
> >   tests: Fix code style on path/filename routines
> > 
> >  Makefile.am                                |  23 ++-
> >  protocol/weston-test.xml                   |  16 ++
> >  tests/internal-screenshot-test.c           | 233 +++++++++++++++++++++++++++
> >  tests/internal-screenshot.ini              |   3 +
> >  tests/reference/internal-screenshot-00.png | Bin 0 -> 5149 bytes
> >  tests/weston-test-client-helper.c          | 164 +++++++++++++++++++
> >  tests/weston-test-client-helper.h          |  40 ++++-
> >  tests/weston-test.c                        | 243 +++++++++++++++++++++++++++++
> >  tests/weston-tests-env                     |  18 ++-
> >  9 files changed, 728 insertions(+), 12 deletions(-)
> >  create mode 100644 tests/internal-screenshot-test.c
> >  create mode 100644 tests/internal-screenshot.ini
> >  create mode 100644 tests/reference/internal-screenshot-00.png
> >