[Libreoffice-qa] Test case naming

[Petr Mladek]

Yifan Jiang píše v St 16. 11. 2011 v 14:48 +0800:
> 4. The amount of how many localization is another concern. In the new scheme,
> for each priority we will have "7 components * 4 locale * 4 priority" (112)
> groups. If I understand it right, when there are 10 locales, the number will
> be 280 ?!

IMHO, anything more than 30 testgroups is not usable. It will be hard to
analyze results. It will be hard to create test run for P1 tests...

=> I think that we can't have language name in the 
split test groups by languages. We have 4 languages now. IMHO, it is
realistic to have 20 languages in not that far future.

=> we can have language in the subgroup names only.

> While another choice to be considered could be switching the position
> of test types and components, something like:
> 
>        / p1-en-Feature Tests              - 7 components (en)
>        / p1-xx-Feature Tests              - 7 components (xx)
>        / p1-en-L10n Regresstion Tests     - 7 components (en)
>        / p1-xx-L10n Regresstion Tests     - 7 components (xx)
> master - p1-en-Function Regresstion Tests - 7 components (ATM: multi-language case)
>        \ p1-xx-Function Regresstion Tests /
> 
> Besides it will reduce the groups size to "3*locale*priority", by the fact we
> can't select subgroups when creating test run (I didn't notice it before,
> thanks Rimas), we can create a test run for particular test types, which might
> be more likely to happen than creating test runs for particular components.
> 
> It would be more clear of the strategy by comments Rimas' origin plan:
> 
> >* create testruns based on priority, locale, component, or any
> >combination of these
> 
> -> create testruns based on priority, locale, test type, or any
> combination of these

I am afraid that it is not the effort. We need a compromise. We need to
estimate what test runs will make sense. I am afraid that priority is
the most important criteria. Anything else would crate too many test
groups and is not worth the effort. Please, see the mail "Test Structure
in Litmus" for more details.


Best Regards,
Petr