[Libreoffice-qa] Test case naming

[Rimas Kudelis]

Hi Petr,

2011.11.15 17:30, Petr Mladek rašė:
> Rimas Kudelis píše v Út 15. 11. 2011 v 13:43 +0200:
>>  Also, maybe I shouldn't look that far into future, but I hope that
>>  there will come a day when proper localization of testcases will be
>>  possible (that is, instead of creating a clone of testcase X in
>>  another language, we would actually be able to translate testcase X
>>  into that language). With that in mind, current testgroups (which
>>  represent different locales) would become unneeded.
> I am not sure if the l10n tests will be pure word-to-word translation.
> Some things are done completely different in different languages, for
> example the layout of the letter template, right-to-left language
> features, decimal number delimiter, dates. I am sure that some
> languages would need special tests.

I didn't say they have to be translated word-to-word. They should be
*localized*, and I would expect a localized testcase to suggest
localized number and date formats and other stuff.

RTL, on the other hand, might probably need a few additional testcases.
Though not many, I would guess.

> BTW: How is the localization used during test run?

It's not. It's just shown in the test results.

> I know that I select locale in the "Run Tests - Your Chosen Test Run"
> dialog, see
> https://wiki.documentfoundation.org/Litmus/Litmus_User_Guide#Enter_Test_Configuration
>
> If I select "de", will I see only the "de" test cases from the l10n
> branch? I am not sure if it is important but it would be nice.

No, currently you will still see all testcases.

My idea with localizable testcases though is pretty much about this. If
testcase X was localized instead of duplicated, it would only be shown
once, in user's preferred locale (I don't know yet how exactly the user
would "prefer" that locale though and what a user who would want to test
more than one locale would do).

>> My suggestion is to have a single branch, but carry Priority, Locale and
>> Component information in the Testgroup, and to represent test type by a
>> subgroup. That is, what is now a branch, would become a subgroup, and
>> everything else would become a testgroup (see the attached PDF file).
> Hmm, I am afraid that we would get too many testgroups. It produces 7
> (General, Base, Calc, Draw, Impress, Math, Writer) testgroups for one
> language. We have 109 localizations in sources => we could end up with
> more than 700 test groups.

Which I guess is not realistic, at least in the short term.  ;) Right
now we only have four languages into which testcases are translated.
With 50, it's of course a different story, but with 4 to 10... it's
probably still manageable enough.

>> This allows to:
>> * create testruns based on priority, locale, component, or any
>> combination of these
> The question is what test runs we will want to create.
>
>> * create a single catch-all testrun for a single version of LibO
> would be nice
>
>> * share the same General Functional Tests subgroup between testgroups
>> designated for different locales (that is, you would create this
>> subgroup once, and add it to all locale groups)
> I am not sure what you mean with this. Could we share subgroups between
> test groups in Litmus? Is it clearly visible or is hard to maintain and
> see?

Like I mentioned before, when you create/edit a testgroup, you can add
the subgroups you want to it. So subgroups can be shared, yes.

> BTW: What do you mean with the "Basis Functional Test"? We do not have
> this subgroup in the current structure.

Don't we?
https://tcm.documentfoundation.org/manage_testgroups.cgi?testgroup_id=58

>> * drop 75% of the testgroups (there would be about 28 of them initially)
>> if/when proper testcase localization is implemented, still not rendering
>> the remaining testgroups useless.
> I am not sure if we really could drop them. Well, the question is if we
> need to split between lang-dependent and lang-independent on the top
> level.

I don't think there are many lang-dependent testcases. Desired result
may depend on the language, yes, but the testcase itself?

> Heh, it seems that it is quite complex problem. I am going to write
> another mail where I will try to summarize some ideas :-)

Good luck with that. :)

Rimas