<html> <head> <meta content="text/html; charset=utf-8" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <div class="moz-cite-prefix">Hi Eric and all, Thank you for describing piglit. I also think that piglit is an overkill for the xcb-test. And the lack of API-stability is suboptimal.[1] I think assert-based is a good direction, but cppunit improves that in a lightweight and easy-to-use way: The CPPUNIT_ASSERT does not terminate the program but just outputs an error message, and continues running the subsequent tests. You can run the whole testsuite as a single executable, and select individual testcases etc. There are GUI-based and text-based user interfaces for the same test-suite. Also, there's no mix of programming languages, which makes stuff easier in general. (Everything is C/C++.) Good point about the need for data comparisons. I guess that an easy way would be to just call a comparison function in a cppunit assert: like: CPPUNIT_ASSERT( IsPixmapEqualToPNG(pixmap, "test123-ref1.png") ); Chris [1] We need to think economical here, too. Everybody's time is valuable, and when we are more efficient we can write more testcases or improve xcb. On 2017-03-31 22:25, Eric Anholt wrote: </div> <blockquote cite="mid:87d1cxkxc6.fsf@eliezer.anholt.net" type="cite"> <pre wrap="">Christian Linhart <a class="moz-txt-link-rfc2396E" href="mailto:chris@DemoRecorder.com"><chris@DemoRecorder.com></a> writes: </pre> <blockquote type="cite"> <pre wrap="">Hi Eric and all, I have thought a bit about xcb/test. I suggest the following: * do xcb/test as a fresh project ( i.e. not clone xcb/demo) * And use a regression test framework for it. I propose to use cppunit. ( <a class="moz-txt-link-freetext" href="https://cgit.freedesktop.org/libreoffice/cppunit/">https://cgit.freedesktop.org/libreoffice/cppunit/</a> ) * As far as possible, the tests should run and check themselves automatically without manual intervention needed. Ideally, we'll succeed to have everything running automatically, of course. What do you think? Chris P.S.: Thank you for the info for how to start the process of creating a new project. I'll do that soon. </pre> </blockquote> <pre wrap=""> For running the X Test Suite and rendercheck, I'm using piglit as a testing framework. What you get from that is the ability to write some python to generate a list of test binaries to be run, a framework that executes them, collects their results, and lets you compare results between runs (including image captures, for rendering tests). Comparison is of more relevance to GL implementations where you may not ever pass all your tests, than to unit tests for things like the XCB or the X Server where we intend to actually be entirely correct according to our tests. The downside is that the piglit framework moves pretty fast and our X stuff frequently gets broken. For smaller tests, I've been basically satisfied with just bare binaries with assert()s, like the X Server's test/misc.c. </pre> <fieldset class="mimeAttachmentHeader"></fieldset> <pre wrap="">_______________________________________________ Xcb mailing list <a class="moz-txt-link-abbreviated" href="mailto:Xcb@lists.freedesktop.org">Xcb@lists.freedesktop.org</a> <a class="moz-txt-link-freetext" href="https://lists.freedesktop.org/mailman/listinfo/xcb">https://lists.freedesktop.org/mailman/listinfo/xcb</a></pre> </blockquote> </body> </html>