[Libreoffice-qa] Issues with 3.5.0 - ready for release ...

[Caolán McNamara]

On Tue, 2012-02-07 at 20:32 +0100, Dag Wieers wrote:
> Do we 
> have tests doing automated QA (conversions) to test export filters and are 
> we using fuzzing techniques to test our import filters ?

I've personally been to-date more interested in automated import filter
tests, typically various binary file formats. You can find the existing
tests in e.g. sw/qa/core/filters-test.cxx where the minimal
always-tested test-cases are in e.g. sw/qa/core/data/*/ 

The idea of these is that you use something like
bin/get-bugzilla-attachments-by-mimetype to slurp down a pile of extra
documents and shove them into sw/qa/core/data/*/indeterminate and just
run "make" in sw to see if any of them triggers a crash (or a valgrind
warning when export VALGRIND=memcheck make is used).

Same basic "see if file format parser crashes" idea implemented for
other modules spread around the place, e.g. sc, sd, svtools and stuff
like .xls, .ppt .wmf and so forth.

calc has extended things a bit further for some extra tests to ensure
that selected test documents have the expected calculation results and
Markus demoed at FOSDEM the in-progress stuff for sd to ensure that
selected test documents render as expected.

> The conversion problems have not been reported yet, but I am sure if we 
> throw more example documents (maybe from Google ?) at it we'll find a lot 
> more problems in an automated fashion.

Sounds good, might need to poke existing code a bit to add extra file
formats if following the existing approach. I can help out what that if
any help is needed there.

C.