[Libreoffice-qa] Annouce: Bibisect for MACOSX

[Robinson Tryon]

On Mon, Dec 2, 2013 at 1:02 AM, Norbert Thiebaud <nthiebaud at gmail.com> wrote:
> On Sun, Dec 1, 2013 at 3:52 PM, Robinson Tryon
> <bishop.robinson at gmail.com> wrote:
>>
>> automatically, I hope? The QA Team has been expecting that daily
>> builds would be available promptly in bibisect repos.
>
> semi-automatically... that is automatically but with human
> supervision... has mishap need to be caugh and fixed early
>
> I imagine that there would be a log of a few days... maybe a week
> between the head of a branch and the bibisect repo..

Hmm...a lag of a week in the bibisect repo could be a deal-breaker for
QA members who try to test with daily builds, especially the members
who build LO on their own machines right now.

> if for no other reason that git gc does a better job if it has many
> loose object to compact, compression wise
> and doing git gcc --aggressive to balance that once in a while is
> prohibitively expensive... and as the repo become build, won;t even
> run on most machines

Could we limit our use of 'git gc' to once a week?

>>> This will also allow.. in case a particular patch made the thing
>>> unbuildable for a while... to retrospectively re-build that section of
>>> the history using a 'patched' version at each step to avoid the the
>>> bug the prevented the build to start with...
>>
>> When you say "re-build," if we're re-building from source, how would
>> the tarballs help us?
>
> because I can rebuild _some_ missing section... and stich the rest
> using the tarball
> I've been doing that dance for the past 3 weeks building the Mac bibisect

As I understand it, we use the content of each tarball as input for a
commit in the bibisect repository. Is it merely *easier* to "stitch
the rest" together when operating on separate tarballs than on the
content of commits in a git repo, or is there actually some important
information in the tarball that doesn't make it into the commit?

>> My hope with bibisect is that we'll prevent any of the primary builds
>> from breaking "for weeks at a time." I think that there are enough
>> Win/Mac/Linux people on the QA Team willing to test against the daily
>> bibisect repos that we'll notice if the build breaks for even a couple
>> of days.
>
> no it won't. we already have tinderbox that send email when stuff
> breaks..

Tinderboxes do send email, but AFAIK those emails only go to tinderbox
owners or to devs. QA would like to be kept further appraised about
not only builds succeeding, but also whether those builds can run and
function properly.

> but sometimes some tb are out-of-order

>From what I understand, we did not have official TDF tinderboxes for
all major platforms until very recently. As a result, it was difficult
for QA to maintain reliability with volunteer tinderboxes that might
disappear for a while.

Official TDF tinderboxes are a great improvement in reliability for
us, and will hopefully enjoy great uptime.

> sometimes stuff get
> overlooked, sometimes shit happens etc...

Oh yes -- there are always unexpected problems. If we plan our
workflow carefully, I think we can minimize the affect these problems
have on our testing and QA process.

> iow bibisect will not help us to detect that something does not
> build... sicne we build far more often than we aggregate bibisect
> build...

I believe our latest plan is to add a new build to the bibisect repo
daily (or every 50-60 commits). That will give us pretty good
granularity to track-down the introduction of a regression.

I agree that bibisect is not the best tool to detect if a tb is not
currently building. Aside from emailing out, what else can we do to
more promptly triage and address the situation when a (previously
building) tb starts to fail to produce viable builds?

Thanks,
--R