gerrit-buildbot master builds

[Norbert Thiebaud]

On Mon, Mar 25, 2013 at 3:45 PM, Bjoern Michaelsen
<bjoern.michaelsen at canonical.com> wrote:
> Hi,
> On Mon, Mar 25, 2013 at 01:04:56PM -0500, Norbert Thiebaud wrote:
>> 1/ I have been working on a new script that can monitor both in the
>> same instance...
>> iow do tinderbox and/or gerrit build as needed, in one script instance
>
> Yes, but I want to get rid of the legacy 'tinderbox builds' as much as
> possible,
why ?

> so I want all builds be scheduled through the queue with the builder
> being ignorant what kind of build he does. Make the client as dumb as possible.

Not a good idea. tinderbox build are progressive so you can take
advantage of incremental build. things you cannot do with a gerrit
build since the base of a gerrit patch can be all over the place...
and incremental build kind of work when moving progressively
forward... but is not expected to work if you start to checkout random
point in master

basic tinderbuild can be done without any kind of authorization or
external setup... whereas gerrit based build need a registered builbot
user in gerrit.
iow a much higher barrier of entry.

>
>> no cron job required
>
> I see the cron job as a feature, not a bug. The tb client should be as dump as
> possible, thus ideally should only get a tree to build and then go for it. So
> the logic for this needs to be in the scheduler -- except that David had qualms
> to add such domain specific stuff to the scheduler and I kinda agree. So a
> cronjob watching and feeding the queue might be a Good in the "do one thing and
> do it well"-category.

I have plan to use cron to 'trigger' build for stuff that we do not
want to continually build.. but the cron job would just touch a
semaphore file so that the one instance of tb that run detect that it
is time to try _that_ kind of build.

so for instance you can have a tb scrip that build gerrit patch +
master continuously... and run a full-lang build once a week and a
valgrind perf regression build once every 3 days... the later two
being triggered by cron using a simple touch <tigger_file>  command

>
>> 2/ you cannot get buildbot dispatch tinderbox build... each tinderbox
>> keep track of where it is and what it has done.
>
> If a bot is 'qualified' to represent a platform on gerrit-buildbot,
> gerrit-buildbot should be allowed to track the state for it (and send the
> status mails for it) as it is inherently trusted to be a valid representation
> of that platform. It makes no sense to tread gentoo Linux and Fedora Linux as
> different on tb but the same on patch verification.

gerrit patch verification are reduced to 1 trusted per platform for
ressource resons... but tinderbox are different.
and tinderbox should be able to be brought online/offline... reseted,
changed etc. at the tb owner discretion...
even be able to run without repporting to the tb web-site

>
>> that allow non-registered tinderbox and a better control of the
>> tinderbox by the operator. so for tinderbuild the 'sever' side is
>> merely collecting result, not distribution task to do.
>
> I dont consider non-registered tinderboxes in this at all -- we might or might
> not find a migration path for them later -- but for now I only focus on the
> registered boxes.

well, 'non-registered' tinderbox are the rule not the exception...
there are only 3 box today that are 'regsitered' in gerrit... the one
that do gerrit buildpatch verification...
that leave 20+ that are 'unregistered'...
All these boxes produce useful result and have a very low barrier of
entry. there is no security risk with them, no ssh key roaming
around... the only vector of attack is an email DoS to the tinderbox
website.
box that will do gerrit patch verification will need to be registered
in gerrit indeed. and we want to have some level of control and/or
confidence in the operation of these boxes... iow we would _rely_ on
them to work well and work reliably... so yes the barrier of entry
will be higher, and eventually I expect most if not all of these to be
TDF operated. but for the rest there is no need or benefit to deprive
ourself of the occasional/part-time tinderbox, or make it more
complicated for people to play with a tinderbox for a while on their
own... without requiering any approval, control or admin involvement.

The other consideration is that the tinderbox system today works
independently of gerrit... so if gerrit were to go down hard for a
significant amount of time... we could fairly easily fall back to a
git mirror and continue working without gerrit for some time...with
still tinderbox verification.

iow: Please explain what is the goal and justification to add
complexity and fragility to something that is not broken.

Norbert