[Libreoffice] Number of days for testing RC

[Petr Mladek]

Hi Jean-Baptiste,

Jean-Baptiste Faure píše v Pá 10. 12. 2010 v 20:02 +0100:
> Hi Petr, all,
> 
> Le 10/12/2010 15:24, Petr Mladek a écrit :
> > Hi,
> >
> > Jean has modified http://wiki.documentfoundation.org/Release_Criteria
> > and increased the number of days for reporting blocker bugs from 7 to
> > 10.
> My first name is Jean-Baptiste :-)

Ah, I am sorry, shame on me.

> > He says that they need to do the complete WE inside the period,
> > see http://wiki.documentfoundation.org/Talk:Release_Criteria
> >
> > Heh, what is "WE"? :-)
> Sorry, it is an acronym for a typical French expression : Week-End ;-)

This is why I proposed 7 days instead 5 days. 7 days have the Weekend
included by definition :-)

> I think we need more time to allow volunteers to do manual tests. They
> do these tests during their spare time so often during the week-end.
> > OOo has only 5 days, see
> > http://wiki.services.openoffice.org/wiki/Release_criteria#Stopper_issues
> > How is it there with building and approving the national builds?
> Perhaps am I influenced by the big number of RC we have raised for OOo
> 3.3.0, but I think that if we take our time to deeply test our release
> candidates, we will have better stable version and enjoyed and not
> discouraged testers.

Hmm, I think that OOo-3.3 release is not a good example:

OOo-3.3 development started on 2009-09-21 (branch for OOo-3.2)
OOo-3.3 feature freeze was on 2009-06-24 (branch for OOo-3.3, feature
                                          freeze)
OOo-3.3-rc1 released on 2009-10-18

So, development took 9 months. Initial stabilization took 4 months. RC
phase takes nearly 2 months and is still not finished.

Note that it was not full 9 months of development because many
developers were busy with stabilizing OOo-3.2 and OOo-3.2.1.

Of course, it was affected by the acquisition of Sun by Oracle but it is
far from the original plan to release minor release every 6 months.

The long cycle is discouraging for many volunteers. They want to have
their work living and used by users. Also it is much easier to fix bugs
shortly after a feature is finished than one year after implementation.


There is an idea of more micro bug fix releases. The published minor
release should be well usable for 99% of normal users. The micro
releases could be done every next month or so. They would include just
safe and reviewed bug fixes and should need not full QA. The second or
third micro release should be well usable for 99% experienced users.


> > Also I think that it does not make sense that every national team would
> > do the whole testing. IMHO, 95% of the functionality is language
> > independent, so most of the testing can be shared and distributed.
> Right. For OOo, NL QA-team do only Release Sanity Test. But the current
> test period on OOo 3.3.0 shows that the last blockers have been found by
> the Community and these blockers could not be found by automatic or TCM
> tests; they have been found by users who have done tests on their
> documents they use in the real life.
> To do that we need time.

I see. Well, I think that even the released OOo version included pretty
annoying bugs and some of them would be considered as blockers. Such
bugs were usually fixed in the micro release x.y.1.

We just need to define the right compromise that would be acceptable for
all sides developers, testers, and users.

Just to get better picture. How much time would you need to finish all
known manual tests?


Best Regards,
Petr