No subject
Sun Nov 13 02:13:30 PST 2011
in a specific order. The ids were originally being there since I got
involved. I think it still makes sense to kept them there mainly
because of the "syncing" problem of test cases in different languages.
For example:
#EN - w001 xxx
is supposed to have the same content with (but in different version of
language):
#FR - w001 xxx
#DE - w001 xxx
#pt-BR - w001 xxx
These give us reasonable information showing which cases are supposed
to be "synced" to each other (they may not have exact same steps of
testing because of the diversity of language settings, but they should
test the same areas). So for current testing organization, I think
these ids are still playing their role in L10N test
branches. Otherwise, syncing of cases could be painful. Meanwhile in
Function Regression testing branch, by the fact we are now using a
single case to host all language versions of test case, it may not
make sense to keep the id any more.
> > I suggest to split test cases into several levels by priorities:
> >
> > P1 - highest: used for very basic tests, e.g. app can be
> > installed; it starts; is able to load/save some test
> > documents; so it a kind of smoketest
> >
> > P2 - high: test very common functionality that is used by most
> > users. e.g. able to write text, insert picture; draw
> > elements; create table; use function in calc; create graph,
> > run presentation
> >
> > P3 - medium: test common functionality that is used by typical
> > a bit experienced office user, e.g. create borders around
> > tables; do animation between slides; modify text style;
> > modify master slide page;
> >
> > P4 - low: test functionality for hi-tech users, e.g. writing
> > macros, using Calc solver, complex operations with data
> > bases
> >
> > I suggest to use the names:
> >
> > p1g - <summary of a P1 global test>
> > p1w - <summary of a P1 Writer test>
> > p2g - <summary of a P2 global test>
> > p2w - <summary of a P2 Writer test>
> >
> > Then we will have all p1 test cases listed before p2 test cases.
> >
> >
> > What do you think?
>
> Prioritizing is probably a good idea, but like I said, random order
> would require some Litmus modification. While certainly possible (and
> probably quite easy), I'm not sure that is what we indeed want.
Actually it is a great idea to have priority here, at least they are
helpful for us to define subset of test runs. For example, we can
create "smoke test runs" by select P1 only test cases when creating a
test run from a full regression branch containing all cases.
That is to say, even before we sort out how order of the test cases
could be implemented, we can always create specific test runs on
demand via the information of the priority "tags". i.e We can define
"smoke test" runs, "basic test" runs, "full regression test" run by
selecting cases, in which all the test cases are divided physically in
test runs and sorting becomes trivial. But it still makes sense if we
can do some hacking in ordering, which could save much of the test
cases selection effort and makes the system more flexible. :)
At the same time, as stated before, for L10N test case branch, we
probably still want test case id until we have some better solution
for translation syncing problem.
Best wishes,
Yifan
More information about the Libreoffice-qa
mailing list