How to test Weston gfx (Re: Weston test suite)

[Derek Foreman]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 31/08/15 03:58 AM, Pekka Paalanen wrote:
> On Tue, 25 Aug 2015 10:43:22 +0300 Pekka Paalanen 
> <ppaalanen at gmail.com> wrote:
> 
>> On Tue, 25 Aug 2015 10:16:17 +0300 Pekka Paalanen 
>> <ppaalanen at gmail.com> wrote:
>> 
>>> Btw. I CC'd you on 
>>> https://bugs.freedesktop.org/show_bug.cgi?id=83985 where we 
>>> have been talking about testing things. I wonder how you'd 
>>> handle some tests being specific to a backend, and some tests 
>>> being generic such that a user (the person doing 'make check') 
>>> could run it against any backend he chooses? If we need such 
>>> things... perhaps backend-specificity comes mostly from 
>>> configuration and options, so maybe we might be able to unify 
>>> the options so much that the differences don't matter.
>>> 
>>> And then screenshot-based tests are likely something we want
>>> to run once for each possible renderer for a chosen backend.
>>> 
>>> 
>>> Hmm... I just got an idea.
>>> 
>>> We've been adding renderer support to headless-backend.so so 
>>> that we could run screenshot-based tests headless. What if... 
>>> we kept headless backend without real renderers (only mocked 
>>> renderer, one capable of initialising 
>>> EGL_WL_bind_wayland_display though), ran Weston/wayland on top 
>>> of Weston/headless, and test clients on the Weston/wayland 
>>> instance?
>>> 
>>> It might require some work on the wayland-backend, but it
>>> would also leave all testing related mocks and hacks into the 
>>> headless backend code, without leaking it into gl-renderer or 
>>> such. And it would stress the wayland-backend which I imagine 
>>> could use more use.
>>> 
>>> Well, that's just a wild idea, I'm not sure what it would 
>>> entail. What do you all think?
>> 
>> Wait... using Weston/headless to host Weston/wayland would 
>> immediately enable software GL rendering, wouldn't it? The EGL 
>> Wayland platform in Mesa today supports software-GL on wl_shm. 
>> This means we should be able to test Weston's gl-renderer on 
>> llvmpipe by just running it with the wayland-backend on another 
>> Weston instace.
> 
> Hi all,
> 
> I think this needs a bit of discussion. Should we be adding more 
> and more renderer features into the headless backend, or should we 
> move to running Weston/wayland on top of Weston/headless for 'make 
> check'?
> 
> It is a blocker question for things like 
> https://bugs.freedesktop.org/show_bug.cgi?id=83985 which Dawid is 
> working on.
> 
> Derek, I recall you had plans to control Weston's repaint loop from
> tests. How would that be affected if we run nested instead of 
> directly on headless?

Right, I'll try to get a new revision of those patches out shortly
after the release...

I can't immediately think of a reason why nesting would cause
problems.  I think it should be ok.

> The reason why I like the nested approach now is that while it 
> allows immediately to run things with llvmpipe, we would not need 
> to modify gl-renderer to e.g. render into an FBO just for testing. 
> We could have the headless backend even initialize a render node 
> and support EGL_WL_bind_wayland_display. Code only related to 
> testing would be limited into the headless backend with its 
> built-in mock-renderer.

Well, I think the clock control stuff would have to take place in the
nested instance, so there's a limit to how much we can
compartmentalize the test stuff to the headless backend?

Screenshots would be taken on the headless backend though - the test
clients would have to connect to both the nested and the parent
compositor?

> What do you think?
> 
> Hmm, and we could pull in fullscreen-shell into the mix and use it 
> on headless. The wayland backend seems to already have code for
> it. This would avoid running weston-desktop-shell or the VKB app
> for the headless instance.

I guess if I was trying very hard to find a reason to dislike this it
would be that changes to fullscreen-shell could break all sorts of
tests that don't intentionally test fullscreen-shell...

Really though, I think it seems like a reasonable idea.  And I think
fullscreen-shell is fairly mature.

Having gl-renderer not have to worry about whether it's in "test mode"
or not sounds like the lesser of two evils.

> 
> 
> Thanks, pq
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJV5KFTAAoJEF5USY5pfxHXhDcH/i0SYW7+9No5Ax/1RtJkZdHt
FW2crauYeH6JzUdIvt7qx3ByXJD3L0JFB27OZFfhfOQ3sccoH7iE2BRu00wNr8ex
5WhA/YpzuJqyQDwyDybi4u9noT/5alJinkItnZphQOZ5l0ly2zvuLSgvdpD6HQnm
AsSqqf6fmd7cEybXM8EzJfnRkNq3CwAzsHE5f15+xv6S2mR9zVj5H/I9/0BjC/y/
X8fBW1dPXAIS+XI8QC4SQA5TtI8k2lHZCNX5tS8JGX8q07o8Hy1/ajt7JXtv5GXt
RLEOyc6nsHxEldlf6eikdNeawiLaCWD9oaGcW9Yd47yweOqncnxedvnLQFCqMuA=
=12qO
-----END PGP SIGNATURE-----