[PATCH weston 1/5] tests: always build tests

[Sam Spilsbury]

Quick thought: there's also an important psychological effect to
building the tests on a standard make because it promotes them to the
same importance as the rest of your code. They become less of an
afterthought and it promotes greater care around how people design the
tests (eg, making the tests clean, making sure they run quickly), as
well as how the rest of the codebase interacts with the tests. We
observed a similar effect at Canonical between the projects which had
test building on by default as opposed to those that did not.

It all depends on whether or not the tests are there as a basic safety
line for managing releases or whether or not tests are used as a tool
to iterate and improve quality. In the latter case, building them by
default is a very sensible decision indeed.

Regards,

Sam.

On Wed, Sep 11, 2013 at 4:53 PM, Peter Hutterer
<peter.hutterer at who-t.net> wrote:
> On 11/09/13 17:32 , sardemff7+wayland at sardemff7.net wrote:
>>
>> On 11/09/2013 07:58, Peter Hutterer wrote:
>>>
>>> check_PROGRAMS and friends are only built during make check.
>>
>>
>> Which is perfectly fine.
>>
>>
>>  > Which is a
>>>
>>> great way of introducing compiler errors in tests.
>>
>>
>> Agree, but we should fix the workflow, not some arbitrary “problem”.
>
>
> from my experience, every project I've worked on that has a test suite
> needed this patch eventually, there's always a way to forget to run make
> check and suddenly you find out that it's been broken for months.
> (this is largely because test suites have a tendency to become outdated and
> useless, but...)
>
>
>>  > Always build them, TESTS
>>>
>>> defines what's being run during make check.
>>
>>
>> That’s wrong.
>
>
> how so? TESTS defines what's run during make check. check_* defines what's
> built, the two are related but not the same.
>
>
>> The check_* vars are meant this way to avoid forcing
>>
>> test-only dependencies if you disable tests and to allow one to test her
>> code *before* updating the tests.
>> Packagers tend to “fix” that the other way around (moving tests from
>> noinst_ to check_) quite often…
>
>
> I know the principle, I'm claiming that without this, tests will eventually
> break unless you find a way to force everyone to run make check.
>
> Which, coincidentally, wastes maintainer time too if you get a patch that's
> fine but breaks make check and you have to get through another revision.
>
> Cheers,
>   Peter
>
>
>
>>
>>> ---
>>>   tests/Makefile.am | 12 ++++--------
>>>   1 file changed, 4 insertions(+), 8 deletions(-)
>>>
>>> diff --git a/tests/Makefile.am b/tests/Makefile.am
>>> index 82bf630..398a275 100644
>>> --- a/tests/Makefile.am
>>> +++ b/tests/Makefile.am
>>> @@ -29,18 +29,14 @@ clean-local:
>>>   export abs_builddir
>>>
>>>   noinst_LTLIBRARIES =            \
>>> -    $(weston_test)
>>> +    $(weston_test)            \
>>> +    $(module_tests)
>>>
>>>   noinst_PROGRAMS =            \
>>>       $(setbacklight)            \
>>> -    matrix-test
>>> -
>>> -check_LTLIBRARIES =            \
>>> -    $(module_tests)
>>> -
>>> -check_PROGRAMS =            \
>>>       $(shared_tests)            \
>>> -    $(weston_tests)
>>> +    $(weston_tests)            \
>>> +    matrix-test
>>>
>>>   AM_CFLAGS = $(GCC_CFLAGS)
>>>   AM_CPPFLAGS =                    \
>>>
>>
>>
>
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel



-- 
Sam Spilsbury