[Libreoffice] minutes of tech steering call ...

[Cor Nouws]

Hi Norbert, all,

(sorry for the delay, but of course we all have our mail archives in 
order ;-) )

Norbert Thiebaud wrote (26-06-11 00:59)
> On Sat, Jun 25, 2011 at 4:52 PM, Cor Nouws<oolst at nouenoff.nl>  wrote:

>>>> The more QA, people, the faster it goes, of course. But that is another
>>>> matter.
>>>> And of course, QA has to be a continuous state. So a possible longer time
>>>> from freeze to release, would IMO not mean that we advise people to start
>>>> later with testing :-)
>>>
>>> Then the question is How would 'freezing' improve that (motivate
>>> poeple to do early testing)
>>
>> Not earlier compared to the moment of freezing. But earlier compared to the
>> moment of releasing. Thus more time available for that specific testing.
>>
>
> Ok, let me try to reformulate the problem:
>
> Formal testing of a Beta/Rc require an large amount of work just to
> regression test things... A lot of these tests are not expected to
> yield bug, especially after the first Beta (that is, regression would
> be mostly found at that point, and hopefully very few would be
> introduced by subsequent fixes)
> So a test window of 1 week, will be use mostly doing that and not as
> much used to discover new bugs and test new function exhaustively.
>
> Hence the proposal become: make the release rate of Beta/RC slower.
> and 5 beta/RC at two week interval would yield better result than 10
> Beta/RC at one week interval.
>
> Do I get that right ?

Hmm, that is indeed an interesting and important extending of my comments.
My comment simply is: when you have not enough people to do the QA-work 
in n weeks, make sure you have 2*n weeks (by releasing first beta 
earlier, not by releasing final later).

What you write is extremely important too:
if you need more than one week to do a lot of release tests, a new beta 
in just one week is not useful. (Even worse, it will frustrate testing.)

> If that is indeed the crux of the argument, then I would suggest:
>
> Expand the time between Beta/RC to two weeks (maybe 3 for the first
> beta?)

When I look at:
http://wiki.documentfoundation.org/ReleasePlan/3.5
indeed there are 5 time frames of one week (\ Xmas).
So a first frame of 3 weeks, 2nd and 3rd of 2 weeks and then two of one 
week, looks much better.
Maybe the same is valuable for the RC too: allow two weeks for the first 
cycle.
Both for Beta and RC this larger first period is important too, since 
both with Beta and RC new groups of users (i.e. testers) will get 
involved. And setting the new environment up the first time, just takes 
longer.

But hey, we still did not even sum up the criteria that we need to 
decide what we have to do to play a rather safe game this time.

> (I would also suggest release on Thursday so that feedback from
> the second week-end have a chance to be fixed before the next release)

Marketing wise, release on Thursday is risky, because one day later 
(what happens) makes it Friday..

> But then, QA still need to use Daily Build, as much as possible,
> during the period to confirm that bugs found in the current Beta/RC
> are indeed fixed to satisfaction _before_ the next Beta/RC comes out.
> In other words, use a continuous QA process on top of a formal
> Release-Based QA, to limit the number of formal Release-QA while
> maintaining a good pace of fixing...

Fully agree.

Regards,

-- 
  - Cor
  - http://nl.libreoffice.org