[Xcb] xcb/test: I propose to base it on cppunit

[Christian Linhart]

Hi Uli and Eric,

Thank you for your input.

The current tests in libxcb run without display connection as far as I can tell.
Maybe this should remain this way?
E.g. if "make check" is called from a batch-job on a build-server, then it won't have a display connection.

If we create a real testsuite, it might become quite large. Maybe that'll be inconvenient as part of libxcb.
In that case xcb/test might make sense.
On the other hand, if it is included in libxcb, then it'll be run more likely before making a release, and it'll be updated more likely.
What do you think?

The test-framework "check" is lightweight and seems to be easy to use, which is good.
This is better than using bare asserts.
No sure how well this works with larger test suites.
"check" is only one headerfile, which is nice because it can be included in the project, and we'll not have to add an outside dependency.

Piglit seems to be too heavyweight and complicated.
The complexity may be warranted for OpenGL, but we won't need that.
And they are changing stuff all the time, which is suboptimal because it takes energy away which could be used for writing testcases.
So, we seem to have pretty much the same intuition about it, Erik.

Comparing with reference data is also possible with asserts. Just call a comparison function from an assert.

Cppunit is a bit like "check" in that it provides a convenient way to write asserts that don't crash but are just reported.
In addition to that it also provides user-interfaces that let you run only certain testcases of a suite. (There are GUI and text user interfaces.)
I am not sure whether "check" provides that.
Also, like "check", everything runs in one executable. This takes away complexity, which is generally good.
My intuition tells me that cppunit is suitable for larger testsuites than "check".
But I don't know "check" well enough, so I am not sure.

So, I guess that the open questions are:
* "check" vs cppunit
* enhance the testsuite in xcb/libxcb, or create a new project xcb/test?

What do you think?

Chris

On 2017-04-01 11:56, Uli Schlachter wrote:
> Hi,
>
> On 31.03.2017 22:25, Eric Anholt wrote:
>> Christian Linhart <chris at DemoRecorder.com> writes:
> [...]
>>>   * And use a regression test framework for it.
>>>     I propose to use cppunit. ( https://cgit.freedesktop.org/libreoffice/cppunit/ )
> [...]
>> For running the X Test Suite and rendercheck, I'm using piglit as a
>> testing framework.
> [...]
>> For smaller tests, I've been basically satisfied with just bare binaries
>> with assert()s, like the X Server's test/misc.c.
> just some small 5c: libxcb already has some tests and uses "check" for
> them, see https://cgit.freedesktop.org/xcb/libxcb/tree/tests
>
> Cheers,
> Uli