Weston test suite

[Jon A. Cruz]

On 08/24/2015 02:50 AM, Pekka Paalanen wrote:
> On Sat, 22 Aug 2015 00:00:58 -0700
> Bryce Harrington <bryce at osg.samsung.com> wrote:
> 
>> On Fri, Aug 21, 2015 at 10:35:34AM +0300, Pekka Paalanen wrote:
>>> On Thu, 20 Aug 2015 11:05:39 -0700
>>> Bryce Harrington <bryce at osg.samsung.com> wrote:
>>>
> 
>>>> (I wish we had some tests with coverage of this chunk of code, so I could
>>>> give my usual plug for adding test cases; if we did, it'd probably
>>>> pretty easy to add a couple test cases here and give us a bit extra
>>>> confidence.)
>>>
>>> It's a bit tough in this case, I'm not sure if we can affect what
>>> outputs the kernel exposes, so it'd be tied to the test machine.
>>
>> No, the way you'd do it is to construct a "mock kernel", and then
>> whereever the functionality expects data from the kernel (or is making
>> kernel calls), you abstract that so you can slip your mock in instead.
>> Sort of the same idea as how we replaced the graphics card / monitor
>> with synthetic ones in the headless testing.
>>
>> I know it probably sounds like a pain in the ass (and it is), but if
>> you're serious about wanting stronger automated testing of Weston, then
>> that's the sort of stuff that needs done.  And it's certainly
>> understandable to want to put off spending time writing tests, but it's
>> the same thing with documentation and look how horribly undocumented
>> Wayland is - unless you document and write tests as you go, it ain't
>> gonna get done.
> 
> Hi,
> 
> it's actually good to bring this up right now. We have the old test
> suite which is painful to work with, and we have the new test suite
> which is seriously lacking to become on par even with the old suite in
> terms of helpers.
> 
> Jon, what are your plans in continuing with the test suite?
> 
> I haven't seen a new patch series in quite a while, and I don't see any
> pending in Patchwork either.


Yes,

The other project I work on just finished pushing out their latest
version, so Wayland was getting refocused for my attention.

Patches for both base Wayland testing and explicit Weston testing
fixtures should be coming once things are updated to apply to the latest
code.

Also, if we are talking about actual mocking actual system calls, that
can be done but does take a little thought.


If we can pinpoint a few target cases for testing to cover, then getting
the needed support in place can flow a lot quicker. At many other groups
I've been the main person to deliver fixtures and mocks to allow other
people to write tests against them. Ideally we do the heavy lifting once
and then allow individual developers to easily test a much wider range
of functionality and scenarios.

Pragmatically, If I don't make testing easy, then I end up the only one
writing tests. Don't want that case :-)


-- 
Jon A. Cruz - Senior Open Source Developer
Samsung Open Source Group
jonc at osg.samsung.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20150824/4bda08ec/attachment.sig>