[Libreoffice] concept for c++ based subsequenttests

[Bjoern Michaelsen]

On Wed, Nov 30, 2011 at 06:24:19PM +0100, Michael Stahl wrote:
> On 30/11/11 14:59, Markus Mohrhard wrote:
> > The test itself is a bit ridiculous 
> 
> so it is a faithful reproduction of the qadevOOo tests :)

One can consider qadevOOo to be fuzzy testing with a minimal intelligence RNG.

> > - does not need an installation
> > - needs much more makefile lines than the old tests ( there might be a
> > solution to this )
>
> seems makefile is so big that it didn't fit in the mail ;)
> 
> i am somewhat unsure if the current approach of having essentially a
> custom makefile for each test is the right way to go; sure you don't
> need a running office, but instead every test makefile needs careful
> maintenance so all the dependencies are mentioned (which are not always
> obvious), and custom rdbs and stuff.
> 
> now if only we could run soffice from the solver directly...

IMHO the pragmatic solution to this is to run them as subsequentests when
dependencies are getting ugly. Writing them in sane C++ is of course better
that using Junit, but nontrivial tests depending on the full product is not a
big bug IMHO. I think by now pretty much everyone builds a dev-install anyway,
and linkoo makes this no big overhead.

And from a pragmatic point of view this saves you all the manual dependency
generation for nontrivial tests which has been shown again and again to be
fragile and subject to random failure (see recently the sd filters test). Of
course a plain, well-contained test in sal can should be run in the build, but
when you need half the office as a dep anyway, I just dont see the point of
fiddling around with the dep manually.

> i second Stephan's request about not using qa/unoapi: that has a very
> specific meaning firmly tied to the qadevOOo mess.

Can we sort this in with the qa/complex test, i.e. tests written with real
developer intelligence for real world scenarios, maybe?

> hmmm.... i wonder if it would make sense to have UNO API tests written
> in Python: that should be much easier on the eyes than boilerplate-heavy
> C++/Java... and i think there is a need for tests at the UNO API level
> no matter how many core C++ level tests we have, because there is really
> no other way to find regressions in that area (nobody tests that
> manually), might as well try to maximize the productivity...

In general, I agree: Python is the best tool we likely have for this job. OTOH
it is bound to the UNO API and I wonder if they can be written in a way that
they do not all die with:
http://wiki.documentfoundation.org/Development/LibreOffice4 .

Best,

Bjoern