[Libreoffice-qa] Test Structure in Litmus

[Yifan Jiang]

Hi Petr, Rimas and all,

Besides the comments in the bottom, I also did Litmus update yesterday:

    - Master Feature branch

        - The empty branch has exactly the same structure of its Function
        counterpart, since we might also have priority and language specific
        consideration in feature testing. A version-specific branch, in which
        the practical test cases should be populated, can be always cloned
        from this branch.

        @Petr, but do we want a more concise structure for this one?

    - Master Function branch

        - 8 test groups, only test cases in priority 1, 2 were populated, we
        didn't have P3/P4 cases yet. The cases were just categorized to
        priority from my understanding from Petr's comments:

            P1 - highest: used for very basic tests, e.g. app can be
                 installed; it starts; is able to load/save some test
                 documents; so it a kind of smoketest

            P2 - high: test very common functionality that is used by most
                 users. e.g. able to write text, insert picture; draw
                 elements; create table; use function in calc; create graph,
                 run presentation

            P3 - medium: test common functionality that is used by typical
                 a bit experienced office user, e.g. create borders around
                 tables; do animation between slides; modify text style;
                 modify master slide page;

            P4 - low: test functionality for hi-tech users, e.g. writing
                 macros, using Calc solver, complex operations with data
                 bases

        - The name of function test cases are updated without "ID" prefix

        - The name of l10n test cases are updated without "locale" prefix, but
        I still keep the "component#number" prefix (instead in a appendix
        brackets) mainly because the test cases will be sorted by components
        prefix so that all the test cases for the same components canbe
        reviewed together.

        - In P3-L10N Test group, there are some "orphan" german test cases
        because previously the german cases were not exactly synchronized with
        English versions, these "orphan" cases are remained after my manual
        synchronization using google translater. I didn't finish review all of
        them but I'll remove duplicated ones while refer to valuable ones to
        create new English test cases. Please just ignore them, I'll clean
        them up later on :)

On Mon, Nov 21, 2011 at 12:18:36PM +0100, Petr Mladek wrote:
> > IMHO, attach it to test case is the most safe way from the design's view,
> > which gives us the most flexibility of expanding the system.
> 
> I slightly prefer this because of the flexibility. We could switch the
> meaning of any test case at any time without moving it in the structure.
> It would help us to create easier structure.

Yes, I talked to Rimas days before in irc and we also agreed to add the
checkbox to the test case at this phase.

Rimas also mentioned it might be possible to do further hacking to have an all
check box indicated all cases in a group/subgroup are language
dependent/independent, but it is another thing after all.

> > But in UI level, subgroup maybe a better place because the test cases will be
> > created by different people, some of whom may not notice the usage of the
> > checkbox for each test case.
> 
> It might be solved by reasonable default. I would set it as functional
> test by default because most tests should be language independent. It
> should be possible to fix the status at any time, see below.

Yes, it's perfectly fine :)

Best wishes,
Yifan