[Libreoffice] Proposal: slowcheck -- some numbers for sc

[Caolán McNamara]

On Tue, 2011-09-20 at 11:27 +0200, Markus Mohrhard wrote:
> The sc unit tests are slow because we use the filter-tests as an way
> to test not only the import but also the first recalculation. We
> havenow already about 10 files that we import and check and I have
> some additional that are not yet finished.

So, here's my numbers from sc. --enable-dbgutil build, linux, x86_64
I did the make non-parallel, in -jX mode the two sc unit tests would be
run in parallel.

a) all already built, make in sc, i.e. check depends and do nothing

time make -sr build
real    0m4.115s

So there's a 4 second make setup cost there already for me.

b) rebuild one file, i.e. ccache + link

touch source/core/data/attarray.cxx
time make -sr build
real    0m43.462s

Ouch, but as expected, linking isn't very fast.

c) all built, make sc, i.e. check depends and run tests

time make -sr
real    0m11.275s

So there's apparently about 7 seconds spent in the tests, that's quite
long alright, much less than a link, but fairly long.

So I've now added a timer listener to our cppunit launcher to tell us
how long each test takes[1] and indeed it takes about 70ms to load a
.xls, 370ms for an equivalent .ods and about 500ms(?!) for an
equivalent .xlsx

So the problem isn't that loading files is inherently slow but that
loading .ods and .xlsx files is slow [2]

In an ideal world I imagine the best spent effort would be on improving
the import speed for .ods and .xlsx, seeing as that improves the
real-world case too.

As a quick hack it's possibly a fixed overhead cost per filter, so
bundling the various .ods and .xlsx tests together into one file per
format might knock a lot off the unit test time. It may then be the case
that additional content added to a single .ods/.xlsx is a tiny cost
relative to a fixed cost of loading one in the first place.

Or stick to xls for tests, that filter is faster :-)

[1] you need to define TIMETESTS in cppunittester/cppunittester.cxx to
get the timing information, maybe this is worth hooking off some command
line option or env variable ?
[2] in --enable-dbgutil mode anyway

C.