[Libreoffice-qa] Test case naming
Yifan Jiang
yfjiang at suse.com
Mon Nov 14 21:34:29 PST 2011
Hi Petr, Rimas and all,
On Mon, Nov 14, 2011 at 10:25:28PM +0200, Rimas Kudelis wrote:
> >> That is to say, even before we sort out how order of the test cases
> >> could be implemented, we can always create specific test runs on
> >> demand via the information of the priority "tags".
> > BTW: How do you suggest to create the priority "tag"? Is there any
> > better solution than to put it into prefix of the test case summary?
>
> Well, as an alternative, branches/groups/subgroups could be reviewed
> again. :)
The idea is actually brilliant! I also thought a bit in this way, but didn't
conclude anything cenrtain at the moment. The problems I can think by
leveraging branch/group/subgroup is
1. The test cases priority heavily depend on Litmus tool that it doesn't
carry the information with their summaries. Instead, the group/subgroup/branch
where they are in tell us the priority.
2. The maintainence effort and process may be increased a bit, but in a
acceptable amount (at least for me).
The benifits are:
1. Creating test runs can be easier that higher level definition would
allow admins to create a specific priority test run simply by selecting
corresponding subgroup, group or branch of test cases, instead of reviwing
each test case's title.
2. The maintenence of test cases is assumed to be more elegant and
intuitive. For example, we can change a test case's priority by change its
corresponding subgroup, group or branch, instead of changing test case's
title.
The following is something they would look like, assume we don't change a lot
the whole structure of current organization.
Overall satistics:
We now have 3 branches:
master regression branch
- 1 group
- 7 subgroups in each group
master l10n branch
- 4 groups
- 7 subgroups in each group
master feature branch
- we may not consider case priority here?
Case 1. Carry priority information with subgroups name
Prototype:
Branch Master Function Regression Test
Group: Function test
Subgroup p1 writer
Subgroup p2 writer
Subgroup p3 writer
Subgroup p4 writer
Subgroup p1 calc
Subgroup p2 calc
Subgroup p3 calc
...
Case 2. Carry priority information with groups name
Prototype:
Branch Master Function Regression Test
Group: Function test p1
Subgroup writer
Subgroup calc
...
Group: Function test p2
Subgroup writer
Subgroup calc
...
Group: Function test p3
Subgroup writer
Subgroup calc
...
Case 3. Carry priority information with branches
Branch Master Function Regression Test p1
Branch Master Function Regression Test p2
Branch Master Function Regression Test p3
Branch Master Function Regression Test p4
All the structure inside different priority branches are the same.
Branch Master L10n Regression Test p1
Branch Master L10n Regression Test p2
Branch Master L10n Regression Test p3
Branch Master L10n Regression Test p4
All the structure inside different priority branches are the same.
I am slightly inclined to the Case 3, it still keeps most of original things
(groups and subroups), which have already been divided in a clear logic and
not that easy to get familiar with. Also Case 3 is the most easy way if we
want to create different runs in different phase of release.
Intuitively, we can "feel" all the three cases by describe a particular l10n
test case in each:
Case 1 - subgroup.
This is a P1 writer test case in English locale for regression testing
Case 2 - group
This is a P1 English locale writer test case for regression testing
Case 3 - branch
This is a P1 regression test case for writer in an English locale
I feel Case 2 is hard to describe :) Case 1 is okay, but it doesn't really
take advantage against Case 3.
It would be interesting if you share some your ideas of cons/prons of them. :)
Best wishes,
Yifan
More information about the Libreoffice-qa
mailing list