[Libreoffice-qa] test cases quality; was: Ubuntu/Canonical doing more manual testing for LibreOffice?

[Sophie Gautier]

Hi Petr,

On 02/03/2012 16:03, Petr Mladek wrote:
> Sophie Gautier píše v Pá 02. 03. 2012 v 14:33 +0100:
>> - are the tests on Litmus good enough or should I write more?
>
> Great point. We have only very few test cases in Litmus (less than 50)
> and the quality is debatable :-/
>
> For example, I see test cases:
>
> 	+ create empty Writer document
> 	+ create empty Calc document
>
> I do not think that we need manual tests for this. This basic operation
> is part of any other complex test. In addition, exactly this is tested
> within few seconds using the smoketest.
>
>
> Another bunch of tests sounds like:
>
> 	+ Translation check of creating a new database
> 	+ Translation check when creating a table in a database
> 	+ Translation check for Formula Editor

Well, I don't think you get the purpose of what Litmus was done for. It 
was for community testing at large, so very easy and short tests to 
bring interest to the testing. It should have help also localizer to 
test there version. Just as we did by the past and it worked well. Some 
spend only 30mn others more that 3 hours because the online tests was 
only the very basis of larger tests with a set of documents. So it's 
more about the life of a team, than only a basic test. Unfortunately we 
don't have the good tool here and no money to develop what could suite 
our needs. Mozilla was developing a tool but it's not yet done either.
>
>
> Of course, we need to check that the application is translated but we
> can't check every dialog manually.

We had that by the past with the VCLTestool.
  Instead of the above particular
> dialogs, we should check that different elements are localized, for
> example:
>
> 	+ "File/New" menu - because it consists of optional components
>                            that are added from xml registry files
>   	+ main menu and one submenu
> 	+ a dialog with tabs, check boxes, combo boxes, itemized list,
>            and other elements
> 	+ help - because it using another technology than the other
>                   dialogs
> 	+ KDE/GNOME safe dialog because they are done another technology
>                   as well
> 	+ extensions - because the translation is done slightly
>                      different way
>
> If one submenu is localized, the other submenus should be localized as
> well if the strings are in pootle.

It's not about localization only (but it's good for CTL and CJK) , but 
also about the design of the dialog that allow to see the whole string 
and then adapt the dialog or the l10n. It's not about to see if it 
works, it's about the quality of the l10n and the design.
>
>
> IMHO, we need to discuss what test cases make sense and create a
> reasonable test cases first.
>
> We are still looking for an experienced QA guy who could step in, teach
> people and drive this forward.

So lets wait for that guy.

Kind regards
Sophie
-- 
Founding member of The Document Foundation