[Libreoffice] QA-hints ? -> question for devs !

[Jean-Baptiste Faure]

Le 13/06/2011 12:47, Yifan Jiang a écrit :
> On Mon, Jun 13, 2011 at 11:47:18AM +0200, Petr Mladek wrote:
>> Also Sophie and Yi Fan are trying to organize testing of beta and rc
>> builds. There is used Litmus server to coordinate the work between the
>> various testers. I am sure that they are looking for more testers and
>> test case writers.
> 
> Hi Cor,
> 
> Great thanks for the message! Sorry I overlooked your mail at first glance,
> our regression testing is organized here:
> 
> https://tcm.documentfoundation.org/
> 
> Here is the instruction how to contribute on running test on it:
> 
> http://wiki.documentfoundation.org/Litmus
> 
> I am probably going to deliver some important new cases or coverage areas to
> the mailing list for review/discuss this week, please keep tuned and let me
> know if you have any ideas about this or about QA.
> 
> Also please do not hesitate to let us know if any questions. We are looking
> forward to you :)
> 
> Best wishes,
> Yifan
> 

Hi Yifan,

I think we should do some simplifications in Litmus.

In fact I do not understand well how branches are supposed to work in
Litmus. When you manage testcases and testgroups, you can see that there
is no one in all branches but only in branch 3.3.1 (in Litmus meaning).
On the other hand when you manage the testruns you can see that each one
contains the three testgroups defined in branch 3.3.1 (EN, DE and FR
testgroups).

It seems to me that it would be more clear and easy to manage if we
defined things as follows:
- branches corresponding to x.y versions of LibreOffice: 3.3, 3.4, 3.5,
and so on.
- the branch x.y+1 would be defined by cloning the x.y branch and next
adding new testcases needed by QA of the new functions implemented in
the x.y+1 version.
- we manage RC and bugfix versions (x.y.1, x.y.2, etc.) only by creating
dedicated testruns: there is no reason to change the testcases set when
we pass from version x.y.0 to rc x.y.1 rc1 to bugfix x.y.1 to ... etc.
So there is no reason to have a new branch for a new rc or bugfix
- we need to define the buildID for each testrun, which does not seem to
be the case at this moment.

According to that, we would have:
- 2 branches 3.3 and 3.4
- for the branch 3.3 :
	+ one testrun 3.3.3-rc1
	+ one testrun 3.3.3-rc2
- for the branch 3.4 :
	+ one testrun 3.4.0
	+ one testrun 3.4.1-rc1
	+ one testrun 3.4.1-rc2
	+ etc.
	+ one testrun 3.4.2-rc1
	+ etc.
- then a branch 3.5
	+ etc.

I think we need to agree on the general organization before
that the system has become too big and too widely used.
I can do these modifications if we have a general agreement.
As that assume to remove several branches and to rebuild testruns, it
would be prudent to not be several to do changes at the same time.

What do think about this proposal? Did I overlook some point which make
my beautiful organization completely stupid?


Have a nice day and sorry for my bad English.
JBF


-- 
Seuls des formats ouverts peuvent assurer la pérennité de vos documents.