minutes of ESC call ...

[Stephan Bergmann]

On 05/29/2015 11:12 AM, Norbert Thiebaud wrote:
> On Fri, May 29, 2015 at 3:15 AM, Stephan Bergmann <sbergman at redhat.com> wrote:
>> On 05/29/2015 05:39 AM, Norbert Thiebaud wrote:
>>>
>>> you mean beside the fact that make check fails on Windows _and_ Mac ?
>>
>>
>> If it routinely fails for you in a specific setup, then please spend effort
>> on either solving it yourself or getting somebody else to solve it.
> I do not have a 'specific setup' but 6 or 7, across 11 slave boxes,
> and 2 release boxes.
> I just can't play wake-a-mole with all of them.

No idea what kind of whack-a-mole is involved here.  If somebody has 
access to a setup where "make check" routinely fails in a specific way, 
they should please invest effort into getting that failure addressed.

> I want to put as much test coverage as I can in gerrit-build and
> jenkins-tb.. but reliability of the tests is paramount here.
> ci that 'routinely' heisen-fails is worse than no ci at all.

That is a different topic than Björn's "there is no excuse not to run 
'make check' anymore at least before pushing."  If the sporadic failures 
mentioned below get in the way of some process, then that may be a 
reason not to run "make check" as part of that process.

But from my personal experience at least, the sporadic failure rate is 
low enough in practice to make manually running "make check" feasible 
before a push (and re-running in case one hits a sporadic failure).

>> Unfortunately, some of the tests sporadically fail, independent of platform.
>> It is hard to remove all the non-deterministic races from the code base.