[Libreoffice-qa] Test case naming

[Petr Mladek]

Hi,

The list of existing test cases looks the following way in Litmus:

id   # testcase summary                                      # subgroup
-----------------------------------------------------------------------
1066 # g001 - Installing LibreOffice                         # General
1061 # g002 - Uninstalling LibreOffice                       # General
1053 # g003 - First launch of LibreOffice                    # General
1063 # g004 - Creating a new document                        # General
1048 # w002 - Importing MS Word documents                    # Writer
1052 # w003 - Exporting ODF text document to MS Word formats # Writer 
1049 # w004 - PDF export of text docs                        # Writer
1054 # w001 - Creating a new text document                   # Writer

See also the screenshot at http://download.go-oo.org/qa/test-case-list.png


I see two problems here:

1. The test cases are ordered by the id, so the first writer test case
   is the last in the list.

   I wonder how complicated would be to hack Litmus to sort it by
   the testcase summary.


2. I am not sure what is the meaning of the numbers 001, 002, 003.

   It looks like they define the order in which we should process the
   test cases. If this is true, it does not look ideal:

	+ if we do another important test case, we will need to rename
          all less important test cases to keep the right order

        + test cases will be checked by many people; we can't force them
          to do it in an exact order; the result would be that all
          people will test the same test case in parallel

          Hmm, we need to encourage people to do the test cases in
          random order. We still should somehow prioritize the test
          cases.

    I suggest to split test cases into several levels by priorities:

    	P1 - highest: used for very basic tests, e.g. app can be
             installed; it starts; is able to load/save some test
             documents; so it a kind of smoketest

        P2 - high: test very common functionality that is used by most
             users. e.g. able to write text, insert picture; draw
             elements; create table; use function in calc; create graph,
             run presentation

        P3 - medium: test common functionality that is used by typical
             a bit experienced office user, e.g. create borders around
             tables; do animation between slides; modify text style;
             modify master slide page;

        P4 - low: test functionality for hi-tech users, e.g. writing
             macros, using Calc solver, complex operations with data
             bases

    I suggest to use the names:

    p1g - <summary of a P1 global test>
    p1w - <summary of a P1 Writer test>
    p2g - <summary of a P2 global test>
    p2w - <summary of a P2 Writer test>

    Then we will have all p1 test cases listed before p2 test cases.


What do you think?


Best Regards,
Petr