[Libreoffice] 3.5 release from QA to point-zero

[Norbert Thiebaud]

On Tue, Sep 13, 2011 at 2:42 PM, Cor Nouws <oolst at nouenoff.nl> wrote:
>
> Thus the solution is: earlier feature freeze.

I disagree that this is _the_ solution. we could feature freeze
today... and that would not change a things unless QA is actually
done... and reciprocally, there is no need for a feature freeze to
start doing serious QA work... that is, even if a couple of feature
were to be integrated in the 2 or 3 weeks you may want to extend the
freeze, that still would translate in 95%+ of the code untouched in
these 2 to 3 weeks.

That is why I mentioned what mmeeks translated into a 'slushy freeze',
that is a period pre-official freeze where we self-impose an elevated
attention to commit breakages or a 'quiet period' if you will, where
emphasis is on bug fixing... in counterpart we need to get QA used to
QA master. Heck I even thing we could function that wall all the way
to RC1 (and use the notion o 'beta' just as a 'state of affair'
buzzword, but still on master) If someone really want to commit
invasive/dangerous things in the pre-RC period, then they should do a
feature branch until we branch for RC1. at RC1 you get 2 full weeks to
do a formal QA campaign, and then go on weekly RC... Again with QA
using daily build to verify asap that bug are closed... If continuous
QA was paid attention to, the formal RC1 QA should be mostly a matter
of verifying that things do still work.

I strongly believe that the best way to have a quality release is to
have a process that tend to make RC1 QA an acceptance test rather than
a 'let's start finding the bugs' starting point, which I feel is
pretty much what happened for 3.4... Granted for the 3.4 release,
master has seen pretty invasive change quite shortly prior to the
official branching for beta ( due to m106 merge mostly), so certainly
the dev side of the house made it very hard for QA to kick in gear in
a timely manner.... but we will _not_ have that again for 3.5.
(actually it has become very very unlikely that we will have any kind
of massive merge anymore. any works pulled from an old CWS or AOOo --
if they start coding something useful -- will necessarily be in the
form of patch series with limited functional scope)

What the QA team can do this time around though, is not hold a grunge
against the mistake that were done in 3.4 and em-brass the goal that
master be 'should deliverable every day' (1).
Of course that is an extreme requirement... but the idea behind this
motto is that we should aim to be in a position such that at any given
day, we could decide to branch a RC1 for the next version.

The elephant in the room to make that (and quite frankly any other QA
approach) a viable reality is automated testing. These are hard, and
even harder when GUI is involved, but they are well worth it.  Stephan
Bergmann just took up the goal of getting subsequent tests running
again into daily build.. that is a great step in the right
direction...


Norbert

(1) that is why I do not give to much weight to the objection about
'moving the freeze date'. We choose a fairly rapid release schedule,
if a feature cannot make it into one release... it will make the next
one... six month later... it is not like missing the cut-off limit
means 2 to 3 years delay anymore...