why a master-tested branch is important and incentives for CI over no-CI (was: test infrastructure ideas appreciated ...)

[Bjoern Michaelsen]

Hi,

On Wed, Jun 10, 2015 at 06:01:54PM -0500, Norbert Thiebaud wrote:
> No today you cannot, as there is no guarantee that the intersection of
> the set of the commit built by each tinderbox is non-empty.
> In fact the odds that that set is non empty during a given work-day is
> pretty low.

Yes, that is the infrastructure challenge in this. However, it should be
solvable, if:

a/ the requirement of updates on the last-known-good-everywhere state is
   roughly 'once in 24 hours' (more frequent updates are nice, but not
   essential)

b/ if tinderboxes are taught to build the latest commit that was marked green
   on another platform, if there is one. Only if there is no such commit it will
   try to create one by building the latest and greatest.
 
> as I said earlier a tag (or a branch which is the same thing
> fundamentally) defeat parallelism.

How so? The tinderboxes would use git-notes as you suggested. Only after that a
cronjob would regulary update a branch to the latest all-green commit to make
the info as easily accessible (the git-notes could still be around). From the
technical point of view, all that is trivial so lets not rathole too much on it
here.

So in total, there are two things needed for this:
1/ Make tinderboxes find at least one commit each day that they all did build
   and test and marked as green.

--> This is a technical challenge. It should be solvable with e.g. git-notes.

2/ Make the last-known-good-everywhere state the default base for new and
   casual contributors in our documentation and culture (allowing those who want
   to work on master directly without CI to continue to do so).

--> This is a _cultural_ challenge. Its rather trivial from the technical side
    once 1/ is done. There might be multiple ways to communicate the results of
    1/, we should use whatever works best to reach the broader audience of new
    contributors.

Anyway: 1/ is the technical/infrastructure challenge to attack first and it
seems we agree on that. So lets not dig too deep into 2/ for now, which is
trivial from the technical side.

Best,

Bjoern