[Libreoffice-qa] Test case naming

[Rimas Kudelis]

On 2011.11.14 12:28, Petr Mladek wrote:
> Yifan Jiang píše v Ne 13. 11. 2011 v 18:46 +0800:
>> For example:
>>
>>     #EN - w001 xxx
>>
>> is supposed to have the same content with (but in different version of
>> language):
>>
>>     #FR - w001 xxx
>>     #DE - w001 xxx
>>     #pt-BR - w001 xxx
>>
>> These give us reasonable information showing which cases are supposed
>> to be "synced" to each other (they may not have exact same steps of
>> testing because of the diversity of language settings, but they should
>> test the same areas). So for current testing organization, I think
>> these ids are still playing their role in L10N test
>> branches. Otherwise, syncing of cases could be painful.

Ah, this makes sense.

> So, the number 001, 002, 003, 004 is a l10n test case number (something
> like bugzilla number). Would be enough to mention it in brackets at the
> end of the test case summary? I mean something like:
>
> p1 - test case summary (w#1,en)
> p1 - another test case summary (w#2,en)
>
> and localized
>
> p1 - test case summary (w#1,en)
> p1 - popis testu (w#1,cs)
> p1 - Testfall Zusammenfassung (w#1,cs)
>
> I know that it is not ideal because it wont be that easy to sort the
> test cases by the id and compare the list. On the other hand, syncing
> localized test cases will not be easy anyway. I think that the bug
> priority is more important sorting criteria
>
> Note that
>
> p1 #EN - w001 test case summary looks confusing to me. There are just
> too many identifiers in the prefix. And it does not help with sorting as
> well.

P1 W01EN would be shorter. Still admittedly quite ugly though.

>>  Meanwhile in Function Regression testing branch, by the fact we are
>>  now using a single case to host all language versions of test case, it
>>  may not make sense to keep the id any more.

Note the testcase still has its real id (used in the database). If
needed, it could be made more visible.

> This way, it would look the same for function regression test and
> localization regression tests. The localization regression test will
> just have some extra identification in the brackets.

Like you said, this would make different testcases harder to associate
with each other. OTOH, I guess only the admins often see them all in the
same place.

>>>>     I suggest to split test cases into several levels by priorities:
>> Actually it is a great idea to have priority here, at least they are
>> helpful for us to define subset of test runs. For example, we can
>> create "smoke test runs" by select P1 only test cases when creating a
>> test run from a full regression branch containing all cases.
> Exactly
>
>> That is to say, even before we sort out how order of the test cases
>> could be implemented, we can always create specific test runs on
>> demand via the information of the priority "tags".
> BTW: How do you suggest to create the priority "tag"? Is there any
> better solution than to put it into prefix of the test case summary?

Well, as an alternative, branches/groups/subgroups could be reviewed
again. :)

Also, Litmus allows marking certain test runs as recommended and shows
them on top. This means that separate P1 testruns could be created and
promoted on Litmus homepage.

Rimas