[ANNOUNCE] : Gerrit Patch Verification Buildbot

[Norbert Thiebaud]

Dear All,

_david_ and I have been working for some time to integrate
tinderboxing with gerrit.

== What is it ? ==

The goal being to test build gerrit patch on Linux, Mac and Windows,
to verify them before they get integrated.

To that end a gerrit plugin has been created that manage a  queue of
build-task to be done and serve these to tinderboxes as they ping th
plugin for work to do.
A new evolution of the tinbuild2 script, named 'tb' has also been
developed to better handle build from multiple branches and possibly
using multiple repos and when that become practical, multiple build
directory, all that within one instance of the script.

The gerrit patch below show what a patch tinderboxing looks like :
https://gerrit.libreoffice.org/#/c/1873/

The new 'tb' script is documented here:
https://wiki.documentfoundation.org/Development/tb


== What does that mean to me ? ==

At this point, we are still in beta-test... we will slowly ramp-up
things...so you will sporadically see more and more of 'Review' like
the ones in the gerrit-patch mentioned above.
So you are welcome to keep an eye on these... but for a little while
there may be some false negative or even false positive, due to bugs
or mis-configuration...
As we get better at it, and as we flush-put the kink, these will
become more and more reliable.

== Where are we going with this ? ==

First we need to flush-out the bugs, we also need to find some
solution to improve the usability of some features (see 'Help Wanted
below')

Second we will start to bring more tinderbox online to process these
build, we will ping some known tinderbox operators at that point.

Third as we bring more hardware online, we will generalize the
criteria to trigger the scheduling of a build.. The exact condition is
not completely decided yet... but essentially a patch will require a
'ack' from a committer, and possibly later extended to the whole
'reviewer' group, to allow it to be test built. The reason for the
necessity of a human control at this phase is to avoid that a
smart-ass pushed a patch that replace 'autogen.sh' by a script that
send the ssh private keys of the unfortunate tinderbox that do the
build to a random server somewhere...

Fourth we will ask people to leave the Verify field of the patches to
the care of the tinderboxes, unless there are pressing reason not to,
and wait until they pass the buildbot verification before pushing them


== What should I do if one of my patch fails ==

There are 3 cases:

1/ you based your patch on an already broken version of master, so
your build fail by virtue that the tree you constructed your patch
upon already failed. In that case you need to re-base your patch,
preferably on a point in master that is 'green' :-) . Btw, when using
gerrit to get your patch in the tree, you do not need to pull -r
contently... do git fetch and checkout a commit from master that you
know is good (see the tinderbox website to find such nugget). and
build your patch(es) upon that point... the final rebase will be done
when the patch is 'submitted' via gerrit... and the odds of having a
re-base problem _then_ are pretty low...  worse case you would have to
re-base locally, fix the conflict and re-push to gerrit.

2/ your patch is at fault... fix it and submit a new version :-)

3/ your patch is fine and the tinderbox is 'wrong'... well, see 2/
again... just in case... and if that is _really_ the case, and you
have convinced others that it is indeed a tinderbox problem, then the
Verify -1 placed on your patch by the tinderbox can be overruled by a
Gerrit admin (or someone that has the proper ACL to run the adequate
command line to 'reverse' the Verify -1 placed by the buildbot plugin,
which is blocking 'Submit').
Alternatively, if you are a committer you can push that patch
directly... Obviously should you be wrong and end-up breaking master,
you will be treated with all the scorn and disdain richly deserved
for breaking the tree and all the patch built upon it :-)


== Is there a timeline ==

There are plenty of factors that would influence the road to
production... including financials ones... so, no, no precise
timeline... but it is hoped to be a 2013 project. :-)


== Help Wanted ==

While the basic component are working and usable... there are things
to be smooth over and plenty of usability items to be dealt with.
The gerrit side plugin is written in Java (since gerrit itself is).

Among the fairly big item to do are:

* log parsing :  right now the log reported to the plugin is basically
the raw text file that is captured by the build... that is not very
user friendly.
We are thinking of leveraging Jenkins log processing plugin to maybe
help with that... or maybe extract and reuse the logic already in use
on the tinderbox server to create short-log... either way, we could
use some help with that.

* status display and schedule queue management. Right now one can see
the schedule queue, with crude tools to 'manage' it from the command
line using ssh gerrit buildbot <cmd>
It would be nice to extend the buildbot plugin to allow it to serve a
webpage that everyone could use to see the current state of affair
(how deep are the schedule queue... where is my favorite patch in that
queue )
and allow 'admin' to influence the queue and fix issues.... (like
removing from the queue patch that was accidentally 'approved' but
should really not be built)

* there are some task around the new 'tb' script see
https://wiki.documentfoundation.org/Development/tb#Help_Wanted for
details.

If you feel like helping with these, ping me (shm_get) or _david_ on IRC.


Norbert