Adding bibisect repository support to tinbuild2

[Norbert Thiebaud]

On Sat, Jul 21, 2012 at 5:48 PM, Robinson Tryon
<bishop.robinson at gmail.com> wrote:
>> TODO: right now we make install and deliver at every build... there
>> may be a reason to do that less often... but the logic to only
>> conditionally do that is not obvious. time based suck,
>
> How much of a time suck are we talking about? The machine I set up to
> test the build machinery is pretty old, so the post-build steps
> (install, copy, git add/commit/push) take up a non-negligible amount
> of time. On a modern fast buildbot, I'd hope those operations would
> take no more than... 5min? 10min?

Sorry I was not clear: I meant that chosing to deliver bibisect on a
time-based (rather than every commit or every n-commit) is not good
imo due to the very unbalanced distributiong of commit over time.
>
>> Note: I presume that the bibisect repo as a branch that match the
>> branch we are building on in the source. we switch to that branch
>> before each build.
>
> Do you mean the bibisect repo *is* for the given branch (perhaps by
> name like "bibisect-repo_3.5"), or the bibisect repo *has* a branch in
> it corresponding to the branch we're building?

*has* a branch

so far bibisect has operated only on master afaict... but it would
make sens to also follow the 'stabilisation branch' (libreoffice-3-x
and even libreoffice-3-x-y  and doing that in the same bibisect repo
than master would allow to bisect the entire history from all master
to the latest commit of a realease branch)

>
>> It is the responsibility of the buildbot maintainer to create new
>> branches as needed (it is not trivial to automatically find the right
>> branch point in bibisect)
>
> By figuring out the branch point in bibisect, do you mean the point at
> which we stop shoving builds into one bibisect repo (or branch in the
> repo) and start shoving builds into a different bibisect repo (or
> different branch)?

I mean at one point on the _source_ git a branch is taken... one
should branch on bibisect at the equivalent commit on bibisect or the
one immediadly before if the commit where it was branch has for some
reason no bibisect commit associated with it

> Obviously it would be a big win for us to have
> seamless coverage of commits -- so if I'm testing 3.6 beta, I can have
> repos for bibisect-3.6-release, bibisect-3.5, etc... as far back as I
> want to go,

Indeed that is the idea :-)

> and have certainty that there are no gaps in commits
> between these repositories.

well there are gap to the extent that not every source commit will
have a bibisect commit associated with it... due to the time it take
to build (there can be multiple source commit pushed while the
tinderbox is doing a build) and because of occasional feature-branch
merge (which bring the whole thing atomically.

>
> bibisect-3.5 would contain all builds from master until the 3.5 branch point.
> bibisect-3.5-release would contain bibisect-3.5 plus all builds done
> on the 3.5 branch.
> bibisect-3.6 would contain all builds from master between bibisect-3.5
> and the 3.6 branch point.
> bibisect-3.6-release would contain bibisect-3.6 plus all builds done
> on the 3.6 branch.

it is much easier to have one bibisect that contain all that (and also
would avoid having to keep multiple repo containing most the same
thing except for a few final commit; The 'master' section of 3-7 will
be much bigger than the 'branched' section. git cloning bibisect will
already not be cheap, but having to do it 3 or 4 time !!!


>> 3/ the optional need to push to a remote bibisect repository.. that I
>> have disable so far. There I think some code to be written to control
>> when and how often this is done
>> that is done in do_push()
>> right now, git push manually  (or with a daily cron) will do the trick
>> Note that not everyone that construct a bibisect repo necessarely want
>> or can publish it to a remote repo
>
> Yep, sounds good. I like the ability to run a tinderbox without it
> pushing data/email externally.
>
>> 4/ I've added some sanity check in tinbuild2, to detect upfront if -x
>> is requested but ARTIFACTDIR is not set or is not a git repo (btw
>> ARTIFACTDIR _has_ to be defined in config/$profile.cfg, since there is
>> no good and safe default value for it (I accidentally ran the code
>> with a default ARTIFACTDIR on my Mac it would have eventually blown-up
>> my ramdisk and that could have taken a while before I noticed)
>
> Fair enough. I'll mention that in the README or in the description of -x.
>
>> Also: I tested it on Mac and it works just fine :-), next I'll try to
>> check on my Windows tinderbox.. It it works fine, I'll deploy it on
>> the ByteMark's windows tb.
>
> So you got the Mac build work without mounting the dmgs? It looks like
> you were able to dump all of the 'hdutil attach' bits, which sounds
> great to me (the less platform-specific bits, the better!)

Yep... apparently make dev-install works on Mac too (one learn every day :-) )
so no need for platform specific code
It seems that we won't need special code on windows either \o/

Norbert

PS: one thing I forgot: due to the code that check that we are on the
'right' branch in bibisect, when you create a new bibisect repo, you
need to create a dummy commit so that the 'master' branch exist. I
typicaly do git init + touch .dummy + git add .dummy + git commit -m
"Initial dummy commit"  (that prolly should be added to the README too