[Libreoffice-qa] Test case naming

[Petr Mladek]

Yifan Jiang píše v Ne 13. 11. 2011 v 18:46 +0800:
> On Sat, Nov 12, 2011 at 10:46:48PM +0200, Rimas Kudelis wrote:
> For example:
> 
>     #EN - w001 xxx
> 
> is supposed to have the same content with (but in different version of
> language):
> 
>     #FR - w001 xxx
>     #DE - w001 xxx
>     #pt-BR - w001 xxx
> 
> These give us reasonable information showing which cases are supposed
> to be "synced" to each other (they may not have exact same steps of
> testing because of the diversity of language settings, but they should
> test the same areas). So for current testing organization, I think
> these ids are still playing their role in L10N test
> branches. Otherwise, syncing of cases could be painful.

So, the number 001, 002, 003, 004 is a l10n test case number (something
like bugzilla number). Would be enough to mention it in brackets at the
end of the test case summary? I mean something like:

p1 - test case summary (w#1,en)
p1 - another test case summary (w#2,en)

and localized

p1 - test case summary (w#1,en)
p1 - popis testu (w#1,cs)
p1 - Testfall Zusammenfassung (w#1,cs)

I know that it is not ideal because it wont be that easy to sort the
test cases by the id and compare the list. On the other hand, syncing
localized test cases will not be easy anyway. I think that the bug
priority is more important sorting criteria

Note that

p1 #EN - w001 test case summary looks confusing to me. There are just
too many identifiers in the prefix. And it does not help with sorting as
well.

>  Meanwhile in Function Regression testing branch, by the fact we are
>  now using a single case to host all language versions of test case, it
>  may not make sense to keep the id any more.

This way, it would look the same for function regression test and
localization regression tests. The localization regression test will
just have some extra identification in the brackets.


> > >     I suggest to split test cases into several levels by priorities:

> Actually it is a great idea to have priority here, at least they are
> helpful for us to define subset of test runs. For example, we can
> create "smoke test runs" by select P1 only test cases when creating a
> test run from a full regression branch containing all cases.

Exactly

> That is to say, even before we sort out how order of the test cases
> could be implemented, we can always create specific test runs on
> demand via the information of the priority "tags".

BTW: How do you suggest to create the priority "tag"? Is there any
better solution than to put it into prefix of the test case summary?


Rimas, Ti Fan, thank you both for looking at it.


Best Regards,
Petr