[RFC] [tests] Running the wayland tests against compositor-headless

[Eoff, Ullysses A]

> -----Original Message-----
> From: Sam Spilsbury [mailto:smspillaz at gmail.com]
>>
>> <snip>
>>
> Ah interesting. I guess the point of the tests is to verify the
> backends themselves as opposed to testing the core compositor (well,
> it tests that as an incident, but I guess its not the point?). Does it
> make sense to have also acceptance tests which really only verify the
> core behavior of weston and not the backends themselves?
>

Well, not exactly.  The tests don't *explicitly* verify or depend on any particular
backend.  But, many of them (client<->server tests) depend on the availability
of input devices (i.e. pointer and keyboard).  As it stands, the backends
are the only parts that provide/initialize those input devices (except the
headless backend, of course).  And, as a side-effect, the backends are the
components that get incidental (i.e. implicit) verification--which is a good thing.

There is no reason why we couldn't create new client<->server and/or
server-only tests to verify other *non-input* related parts of core functionality.
Believe me, we need more of em :-).

> > ...but if it's a problem that the tests themselves can't run on the headless
> backend,
> > then changing those in a way to work on headless, too, might be useful.
> 
> So at the moment - they aren't - well, its possible to run the two
> library based ones. 3 of the client based ones are easy to enable, as
> for the other three, two of them protocol support from the backend,
> the other one (xwayland) doesn't really make sense on headless.
> 

Yes, it would be trivial to enable the input-based tests to run on the
headless backend... however, it's unclear to me whether the headless
backend should be the one responsible for initializing/tracking the
keyboard and pointer inputs.  That kinda feels counter intuitive to the
definition of "headless" (but perhaps not :-/).  Maybe we could have the
weston-test extension module initialize the inputs in an idle loop
callback, for example if the output->model == "headless", then
do:

	weston_seat_init_pointer(seat);
	weston_seat_init_keyboard(seat, NULL);

For the other two, which ones are they and what kind of protocol
support is lacking?  Perhaps there is another way we can get them
to work on "headless"?

> The broader reason I'm looking into this is because I want to see if
> we can eventually export the weston-test module as part of the
> installation as well as the headless and no-op backends so that
> applications can run integration tests against weston to verify that
> they handle input, shell surface changes etc correctly. I'm looking
> into doing some GSoC work on a project where I suspect that the
> wayland backend will not be run all the time, and it will be good to
> have integration tests in place to ensure that it doesn't break
> accidentally because developers may not be able to run it regularly.
> 

The headless backend module gets installed, already.  But if you're
also talking about exporting headless backend header(s), then I'm
willing to bet that won't happen.

As for the weston-test module being exported (module and headers),
that would have to be the maintainer's decision (Kristian; aka, krh)
and/or a consensus from other community contributors.

If nothing else, you can always write your own test extension, too, that
will enable you to do integration testing on your particular application
use-cases.

> So - shall I create and submit some patches to get the tests working
> on headless? I suppose this is a desirable thing right?
> 

I wouldn't be the one to make the ultimate decision on their
inclusions into upstream.  It'd be nice to hear what others think about
this topic, too.  Either way, it wouldn't hurt to send patches for review,
feedback, and discussion... even if they end up getting rejected for a
different solution ;).

> Best,
> 
> Sam
> 
> > <snip>

Cheers,

U. Artie