[Libreoffice-qa] REMINDER: Release 4.0.1.2 from libreoffice-4-0-1 branch

[Petr Mladek]

Hi Dan,

first thanks a lot for testing LibreOffice release candidates. It is
important job.

Dan Lewis píše v Čt 21. 02. 2013 v 09:39 -0500:
> On 02/21/2013 09:16 AM, Petr Mladek wrote:
> > IMPORTANT: There are not two weeks between rc1 and rc2 this time. It is
> > another side effect of the late schedule change when we used two weeks
> > between 4.0.0 betas and RCs. We were not able to shift the feature
> > freeze and final release.
> >
> > This should be the last anomality. There should be always two weeks
> > between all betas and rcs from now on,
> > see https://wiki.documentfoundation.org/ReleasePlan.
> >
> >
> > Best Regards,
> > Petr
>       I read your first link, and the summary on that page is not 
> psychologically sound. Study after study has shown that when people work 
> on a deadline, they begin to work faster as they approach the deadline. 
> (Staying up all night in college when a paper is due the next morning is 
> one common example.)
>       Now we do not have the time between RC1 and RC2 to properly test 
> the latter for potential bugs. So, is it possible that we will have to 
> fix some bugs for 4.0.2 because we did not have the time to test and 
> find it in 4.0.1 RC1 or RC2?

I am a bit confused. :-) In the first paragraph you suggest that most of
the work is done the last night before the deadline. In the second
paragraph, you suggest that one week is not enough. So, what change do
you actually suggest?

>       Will this be the last anomaly? I doubt that very much. Any time 
> there are as many changes in LibreOffice between major versions, you are 
> likely to have anomaly as we just did.

Yes, there might be another anomality if we change the schedule again.
Though, this is the last anomality caused by the last schedule rework.
If I did not make a mistake, all the further 4.0.X, 4.1, 4.2 release
have two weeks between betas and RCs.

>       Beside, it appears that this schedule does not consider how many 
> people are available in QA to test what the developers produce. None of 
> us have unlimited time available for testing. If we are going to have a 
> quality produce, we need the time to thoroughly test the software. 
> Please put that in the schedule too.

This is why we started to use two weeks between between betas and RCs. I
am afraid that we could not do much more if we do not want to lose the
effect of the frequent time based releases.

We are not able to catch all regressions during testing. We could cover
only the most important areas with the current resources. Each
regression or bug is less painful when it can be fixed in the next
bugfix release that is available only few weeks after the previous one.

We currently use the following tricks to reduce potential regressions:

1. All commits for already released product are reviewed by another
developer before pushing. It helps to find potential problems in time.
Another side effect is that it is more expensive, so only important or
trivial fixes pass this process => development in the stable branch is
under control.

2. We create unit tests for many fixed bugs. They are proceed for every
build. So, we are sue that these bugs never appear again in the future.

3. There is the testing done by the QA team. It is important but not
much organized these days. I think that most people just install the
last build and use it for daily tasks, verification of bugs, ... It is
kind of random testing but it covers the basic areas out of box.

IMHO, it would be great to have the result of manual testing a bit more
organizes and documented. Then we could know be sure that something was
really tested. Then we could have better arguments about what time is
really necessary for testing. I am looking forward to have more people
using http://manual-test.libreoffice.org.
See also https://wiki.documentfoundation.org/QA/Testing/Regression_Tests


Regarding 4.0.1 release. It is pity that there is only one weeks between
rcs but there are still two weeks between rc1 and final publishing. All
changes for 4.0.1.2 build need 3 reviewers, so we are very careful what
goes in. There is really minimal chance of regressions between 4.0.1.1
and 4.0.1.2 builds.

I hope that it looks more optimistic now.

Best Regards,
Petr