[PATCH weston] tests: Adding simple waycheck validation tool.

[Jon A. Cruz]

On 11/05/2015 10:17 AM, Pekka Paalanen wrote:
> On Fri, 16 Oct 2015 12:23:47 -0700
> "Jon A. Cruz" <jonc at osg.samsung.com> wrote:
> 
>> On 10/15/2015 01:51 AM, Pekka Paalanen wrote:
>>> I think it's better to land this stuff in pieces than massage a
>>> humongous replace-the-world patch series, if we can tell the pieces are
>>> going in the right direction. The bits here are mostly just following
>>> the existing weston-test-client-helper.c (which the commit message
>>> forgot to mention).
>>>
>>> But yeah, probably makes sense to see how hooking even this stuff up to
>>> 'make check' would happen before landing this. That will be one of the
>>> most interesting things here. This patch as is does not add anything to
>>> 'make check' which is a little awkward for a series adding new test
>>> code.
>>>
>>> As for writing tests in the mean time, people should just ignore the
>>> new framework for now, and write tests as if it didn't exist as long as
>>> it doesn't support what the old framework does.
>>>
>>> This way we can incrementally advance on both fronts at the same time.
>>
>> So, the immediate question would now be as to how I should stage things.
>> Should the changes that will go into 'make check' be a separate patch
>> set or should it be an additional patch in this set?
>>
>> I'd say that running it as a separate set might make juggling git a
>> little easier on me. But not to a degree to outweigh what would make
>> reviewing things easier.
> 
> Hi Jon,
> 
> as I'm not sure how you are going to hook things up to Weston and it
> will be interesting to see how you do it, I would like to have it as a
> part of this series adding waycheck. In my view, you can't even
> demonstrate one without the other: waycheck needs a compositor, and
> seeing how 'make check' starts a compositor for a test needs the test
> client.
> 
> I would very much like to see both pieces at the same time, in case one
> requires changes in the other. It would be nice to keep in mind all the
> various test styles (client, module, ...) we listed in the past, so
> that adding a new case later shouldn't cause a huge refactoring.
> 
> By test styles I mean basically the different cases in
> tests/weston-tests-env script.
> 

One aspect for waycheck is that it could be run against most any
compositor at any time, so allowing for those uses makes it less of a
driving factor and more of a client.

The other tests, however, do need to start weston and with different
configurations. Those then fall into the potential white-box testing
world, while waycheck is more for black-box validation.


I've been going through some general cleanup, but will include the some
of the current 'make check' tests conversions also. Might take a little
bit more, but we definitely want to keep things easiest to review.


-- 
Jon A. Cruz - Senior Open Source Developer
Samsung Open Source Group
jonc at osg.samsung.com