[Libreoffice] [Libreoffice-qa] QA ideas for 3.5 WAS Re: Improving the QA and Release for 3.5

[Yifan Jiang]

Hi Cor,

I hope I didn't interrupt anything :) But here's some of my comments.

1. Thanks for Rimas and Petr's actively participating, what we did in the past
two month to improve the structure of Litmus and now the new stuff is online
and a 3.5.0 beta regression test run is created, in which we are encouraging
people to run the tests.

https://tcm.documentfoundation.org/

There are a few important changes need to notify people, especially for l10n
testers that:

    - we now have the localization test cases merged together with English
    cases, both in content and structure. Then we add "locale-dependent"
    property for each test case.

     https://tcm.documentfoundation.org/show_test.cgi?id=3105

    Specifically, a test result submitted for a locale dependent cases will
    not influence the progress statistics of the test cases running in another
    locale condition. For a locale independent case,the executing result will
    be shared among all tests whatever locales were seleted. For locale
    selection I meant the bottom field here in test configuration details set:

     http://wiki.documentfoundation.org/File:Litmus_chose-de.png

    So it is important to make people informed correct locale selection is
    necessary before submitting test results.

    Also it is important to let l10n people know the translation version of
    test cases are not disappeared, all languages versions of test cases were
    just put together to save our effort depend on how much they related with
    languages.

    - we abandoned the build id selection to avoid duplicating test effort. So
    it is important to make people informed to actively use correct build
    submitting a result in the corresponding test run. For build id selection
    I also meant the field in configuration details set (the screenshot needs
    to update for this...):

     http://wiki.documentfoundation.org/File:Litmus_chose-de.png

    - we grouped test cases with priorities, and it is preferred the higher
    priority test cases are executed before lower ones. This can be seen
    when it comes to test group selection after the above configuration set.

2. We also got the wiki page updated and cleaned up. The important start pages
to understand all related stuff can be:

    http://wiki.documentfoundation.org/Litmus
    http://wiki.documentfoundation.org/QA/Testing/Regression_Tests

3. As for the regression test plan, we got a full regression test related
section in the QA processes page typically to describe what a Litmus admin
should do when there comes a build:

    http://wiki.documentfoundation.org/QA/Processes#Full_regression_test_management

Briefly speaking, we create a single test run for every beta and rc phases,
during whose time slot the test cases inside test runs are supposed to be
executed.

For example, from the release plan, we will have 3 beta builds from 5/Dec to
9/Jan. In this phase, we do NOT create three test runs, instead the only test
run "3.5.0 beta regression test" will be recommended there for nearly a month,
when continuously submitting test result in the latest 3.5 beta build is
expected.

4. Is it possible to emphasize the regression strategy in the bug hunting
session you are planning? :)

Not sure if some of the points answered your original question :) Please let
me know if anyone needs more regression test and litmus related information!
Thank you!

Best wishes,
Yifan

On Tue, Dec 13, 2011 at 11:35:44PM +0100, Cor Nouws wrote:
> Hi Petr,
> 
> Cor Nouws wrote (09-12-11 13:44)
> 
> >On the wiki, I added a section with the draft for the later to create
> >separate wiki page:
> >http://wiki.documentfoundation.org/QA/Improving_QA-Release-3.5#DRAFT_for_page_with_info_for_bug-hunting_session
> 
> Is it possible for you - without wanting to stress you - to tell
> smthg more about the expected planning?
> 
> tjanks a lot
> 
>