[Libreoffice-qa] Test case naming

Rimas Kudelis rq at akl.lt
Mon Nov 14 12:25:28 PST 2011

On 2011.11.14 12:28, Petr Mladek wrote:
> Yifan Jiang píše v Ne 13. 11. 2011 v 18:46 +0800:
>> For example:
>>     #EN - w001 xxx
>> is supposed to have the same content with (but in different version of
>> language):
>>     #FR - w001 xxx
>>     #DE - w001 xxx
>>     #pt-BR - w001 xxx
>> These give us reasonable information showing which cases are supposed
>> to be "synced" to each other (they may not have exact same steps of
>> testing because of the diversity of language settings, but they should
>> test the same areas). So for current testing organization, I think
>> these ids are still playing their role in L10N test
>> branches. Otherwise, syncing of cases could be painful.

Ah, this makes sense.

> So, the number 001, 002, 003, 004 is a l10n test case number (something
> like bugzilla number). Would be enough to mention it in brackets at the
> end of the test case summary? I mean something like:
> p1 - test case summary (w#1,en)
> p1 - another test case summary (w#2,en)
> and localized
> p1 - test case summary (w#1,en)
> p1 - popis testu (w#1,cs)
> p1 - Testfall Zusammenfassung (w#1,cs)
> I know that it is not ideal because it wont be that easy to sort the
> test cases by the id and compare the list. On the other hand, syncing
> localized test cases will not be easy anyway. I think that the bug
> priority is more important sorting criteria
> Note that
> p1 #EN - w001 test case summary looks confusing to me. There are just
> too many identifiers in the prefix. And it does not help with sorting as
> well.

P1 W01EN would be shorter. Still admittedly quite ugly though.

>>  Meanwhile in Function Regression testing branch, by the fact we are
>>  now using a single case to host all language versions of test case, it
>>  may not make sense to keep the id any more.

Note the testcase still has its real id (used in the database). If
needed, it could be made more visible.

> This way, it would look the same for function regression test and
> localization regression tests. The localization regression test will
> just have some extra identification in the brackets.

Like you said, this would make different testcases harder to associate
with each other. OTOH, I guess only the admins often see them all in the
same place.

>>>>     I suggest to split test cases into several levels by priorities:
>> Actually it is a great idea to have priority here, at least they are
>> helpful for us to define subset of test runs. For example, we can
>> create "smoke test runs" by select P1 only test cases when creating a
>> test run from a full regression branch containing all cases.
> Exactly
>> That is to say, even before we sort out how order of the test cases
>> could be implemented, we can always create specific test runs on
>> demand via the information of the priority "tags".
> BTW: How do you suggest to create the priority "tag"? Is there any
> better solution than to put it into prefix of the test case summary?

Well, as an alternative, branches/groups/subgroups could be reviewed
again. :)

Also, Litmus allows marking certain test runs as recommended and shows
them on top. This means that separate P1 testruns could be created and
promoted on Litmus homepage.


More information about the Libreoffice-qa mailing list