[Libreoffice-qa] QA ideas for 3.5 WAS Re: [Libreoffice] Improving the QA and Release for 3.5

[Petr Mladek]

Hi Cor,

I have added QA and devel mailing lists into CC. We should not cook such
an important event in background.

Cor Nouws píše v Út 29. 11. 2011 v 00:06 +0100:
> Hi Yifan & Petr,
> 
> Is it OK for you if I proceed more or less as outlined below?
> I have some additional remarks that I'll take into account.
> 
> I think I'll explain clearly to the people the way to Litmus for if they 
> want to do the structural tests.
> I saw some work on changed naming. Is that already in place?

Heh, we are going to do one more restructuralization in Litmus.

Rimas implemented a nice feature yesterday. We are now able to mark some
test cases as "language" and/or "OS" dependent. This option affects when
the test will be marked as "done" in test run. This way we could get
easier structure and optimize the testing efforts.

Yi Fan is creating documentation at
http://wiki.documentfoundation.org/QA/Testing/Test_Cases_Organization
Also he is going to update the existing test cases accordingly. He wants
to finish this until Dec 6 or so.

Another problem is that we still have only few testcases. We need to
create more. We still could do it during the test phase.

Any helping hands are welcome.


> Cor Nouws wrote (21-11-11 10:59)
> > Cor Nouws wrote (19-11-11 16:56)
> > Goal is (as discussed before) get more people's focus on early testing,
> > rather then with the first RC, and also more people testing :-)
> >
> >
> > I have draft the following schedule with special events:
> >
> > - With the first beta, two day's organised bug-hunting:
> > - Friday 9 and Saturday 10 December
> > - the beta-0, makes sure that reliable builds are there for all
> > - mail all people on l10n lists and ask to spread the news
> > - mail to discuss & users
> > - people available on irc for guidance

It sounds good. Rainer is just afraid that we do not get enough people
attention. He tried to do such events in the past and nobody appeared.
Well, I am not sure how Rainer announced it. You might know better
channels.


> > - during the months, a two day's competition:
> > - Saturday 14 and Sunday 15 January
> > - a prize for the three that post most bug-weight in those two day's
> > - bug weight: points/issue:
> > 1 for normal, 1,5 for major, 2,5 for every blocker issue
> > - severity needs to be confirmed by QA specialists (in max one week)
> > - prizes are e.g. sweater and mug.
> > - QA specialists cannot win a prize
> > (they should just get the sweater and mug without that)

I am not sure about this action. I am afraid that we could get too many
bug reports and it will be hard to sort them, ... Note that we currently
have only very few people doing the bug triage.

Another problem is who would judge the results :-)


> > - with the first RC, two day's organised bug-hunting
> > - Friday 20 and Saturday 21 January
> > - ... (see above for more)

Same as for the first bug hunting session.


When I think about it, we have a chicken and egg problem. I am afraid
that we do not have enough active QA people that would help to organize
such event. On the other hand, such action could bring new people or get
them from the hiding corner.

I would try a well announced "bug-hunting" session. If we get enough
people there, I would skip the competition because it is hard to
organize.

Another question is how to instruct people during the bug-hunting
session. We do not have enough test cases. One possibility would be to
ask them to create them as described at
http://wiki.documentfoundation.org/QA/Testing/Test_Case
Otherwise, it would end in a chaotic testing. Well, it would be better
than nothing.


Best Regards,
Petr