[Libreoffice-qa] test cases quality; was: Ubuntu/Canonical doing more manual testing for LibreOffice?

[Petr Mladek]

Sophie Gautier píše v Pá 02. 03. 2012 v 19:39 +0100:
> > Do we really need to check the functionality in all 100 localizations?
> 
> It's only checked in 5 or 6 language, even less if you look at the poll 
> I've ran on the l10n list.

It is the current state. I hope that more people will use litmus in the
long term.

Also even these 5 group might check 5x more functions if they do not
duplicate the effort. :-)


> > IMHO, if the database opens in English, it opens in all licalizations.
> > We do not need to force 100 people to spend time on this functional
> > test.
> >
> > Do we need to check translation even when the strings were not changed
> > between the releases?
> 
> yes, because the amount of strings in the database is really big and you 
> need more than two eyes to check for the quality.

Yes but I hope that we could do it more effectively. We do not need to
check all strings every 6 months for every release. In addition, the
check is usually done by similar group of people, so too many reviews
does not bring anything new. Finally, if there is anything too bad,
users will report it. If people are happy with a string one year, we
need not spend too much effort on updating it.

LO is really complex. Users report new bugs weeks and months after the
release because some functionality is used less frequently. The more we
could check during the regression tests during beta phase, the better
for the experience of final users.

I agree that reviewing strings is important. I just try to explain that
we could do much more tests when we separate functional and translations
tests.


> > =>  I strongly suggest to separate translation and functional checks. It
> > is very ineffective to test them together.
> 
> you spare some resources, most of the time tests are done by people in 
> their native language. Do you want to run them only in English?

No, the description of the functional test cases should be localized.
Anyone could run it in any localization. The point is that when a French
guy checks that a database can be created, it need not be checked by
German, Italian, American. These other guys could use the time for doing
some other checks.


> > Yes, it is confusing because they mix the translation and functional
> > tests. All I want to say is that it is not effective and we should not
> > go this way.
> 
> Ok lets try without checking for the translation, we can remove the 
> specific directions about language in the test.

ok


> >> Because each team has to adapt the test in his language, the basis in
> >> English doesn't mention every specificities.
> >
> > Yes, but only small part of the functionality is language dependent.
> 
> yes this is why I didn't see it as an issue to check for the language at 
> the same time the test is run

I probably wrote my sentence a confusing way. I mean that only very
small part of the tests depends on the language.

For example, spell checking depends on language.  If German dictionary
is available, it does not mean that also Frech dictionary is available.
So, you need to check that the dictionary is available in each language.

On the other hand, most of the other functionality works exactly the
same way in all languages. For example, creating the database. You just
need to translate the steps into more localizations, so anyone could run
it. It is not important who runs the test. If it works in Italian, it
will work also in Frech, German, Greek, Japan and other languages. They
do not need to check this.


> > I think that we are on the same page. After all, the new test cases are
> > not that bad. My only problem is that they mix functional and
> > translation checks.
> 
> ok, lets see what it brings if we remove this part from the test. But 
> I'm afraid you're speaking of English tests only with no localization of 
> the tests.

No, we already have a support for translating the test cases. I am sure
that we will be able to do it even more cleanly in the future.


Best Regards,
Petr