Release blocker (Re: [PATCH weston v3 00/15] Implement screenshot-based testing with the headless renderer)

[Bryce Harrington]

On Fri, May 22, 2015 at 03:27:39PM -0500, Derek Foreman wrote:
> On 22/05/15 02:34 AM, Pekka Paalanen wrote:
> > Another failure:
> > 
> > $ make check TESTS=internal-screenshot.weston WESTON_TEST_REFERENCE_PATH=~/git/weston/tests/reference BACKEND=x11-backend.so
> > 
> > FAIL: internal-screenshot.weston
> > 
> > test-client: got seat 0x73a6e0 capabilities: 3
> > test-client: got seat 0x73a6e0 name: 'default'
> > test-client: got global pointer 100 100
> > test-client: got seat 0x73abe0 capabilities: 7
> > test-client: got seat 0x73abe0 name: 'test-seat'
> > test-client: got keyboard repeat_info 40 400
> > test-client: got keyboard keymap
> > test-client: got surface enter output 0x73a820
> > test-client: got keyboard modifiers 0 0 0 0
> > test-client: got pointer enter 0 0, surface 0x73ad50
> > Creating client for test
> > Taking a screenshot
> > Screenshot has been captured
> > Loading reference image /home/pq/git/weston/tests/reference/internal-screenshot-00.png
> > Loading reference image /home/pq/git/weston/tests/reference/internal-screenshot-00.png

^ Probably unrelated but it seems odd that it is loading the same
reference image twice.  Smells like a bug.

> > Screenshot different from reference image
> > Clip: 50,50 101 x 101
> > Bytewise comparison inside clip
> > Mismatched image on row 50
> > 50,0:        0       7b  <---
> > 50,1:        0       7b  <---
> > ...
> > 50,374:        0       7b  <---
> > 50,375:        0       7b  internal-screenshot.weston: /home/pq/git/weston/tests/internal-screenshot-test.c:242: inter
> > nal_screenshot: Assertion `match' failed.
> > <---
> > 50,376:        0       7b  <---
> > 50,377:        0       7b  <---
> > ...
> > Screenshot doesn't match reference image in clipped area
> > Writing PNG to disk
> > Test complete
> > test "internal_screenshot":     signal 6, fail.
> > 1 tests, 0 pass, 0 skip, 1 fail
> > 
> > 
> > The written screenshot is completely black. I believe this is still due
> > to the race between weston-desktop-shell and the test client.
> 
> I'm going to try to fix this as soon as I can reproduce it.  So far it's
> passed every time I've run it...

One thing to look at is if all the settings in .ini are taking effect
properly with this backend.  I'd had similar behavior as the above when
startup-animation was enabled.

I also set background-color, in order to have a solid background instead
of an image; I doubt it, but potentially this could be exercising
a different logic path in this backend.

Sorry, this is probably less than helpful.  I'm ignorant about how the
x11 backend works.

Bryce