[poppler] Poppler reg tests

[Carlos Garcia Campos]

Excerpts from Albert Astals Cid's message of mar sep 13 12:27:27 +0200 2011:
> A Dilluns, 12 de setembre de 2011, Carlos Garcia Campos vàreu escriure:
> > Hi all,
> 
> Hi
> 
> > as you all probably know, our current regression test suite are some
> > custom shell scripts and a lot of pdf files that Albert has collected
> > for some years. During the last few days I have been working on
> > improving the regression test, converting Albert's scripts into a
> > small python program to allow everybody to run their own tests. We are
> > not going to upload pdfs to the repo, so we still depend on Albert
> > to make sure a change doesn't actually introduce regressions. 
> >
> > SNIP
> > 
> > There are still things to do, like adding an option to provide documents
> > to skip, 
> 
> Yeah, the documents to skip is important, i have a few files here that cause 
> infinite loops because of errors in things we do not control (like openjpeg) 
> and i need to be able of ignoring those.

For now, as a workaround you could rename the files to skip as something
like pdf_document_i_want_to_skip.pdf.skip, since only .pdf files are
used.

> It would be good to have some sense of how the test is progressing, the 
> problem at the moment is that files are tested in "random" order, so can not 
> use the current file to know how much is left. It would be good either to sort 
> the files or to get a "1 of 1000" text to know how much is left

Yes, it happened to me yesterday too, it would take a bit longer, but
I guess it's better to get information about the tests first, sort it,
and then iterate it to create the refs and run the tests.

> I am not sure the "Test run in X seconds" works, see here
> 
> $ time ~/devel/poppler/regtest/poppler-regtest --utils-
> dir=/home/tsdgeos/devel/poppler/build-old/utils/ --backends=splash run-tests 
> --refs-dir=./refs --out-dir=./out .
> Testing './alumnes_normativa.pdf' using splash backend: PASS
> Total 1 tests
> 1 tests passed (100.00%)
> Tests run in 0 seconds
> 
> real    0m1.101s
> user    0m1.690s
> sys     0m0.097s
> 
> That 0 seconds seem wrong either to the 1.1 real or 1.6 user seconds

The timer only counts the time to create-refs and run-test, no the
whole program, it shouldn't be much different, though. I assumed that
in most of the cases it will take more than once, and if it didn't,
the time spent doesn't really matter, so it doesn't give
milliseconds. Try with a test that takes a bit longer.

> > html backend support, and I'm sure there are a lot of bugs
> > too. So, feel free to comment, or even better provide patches :-)
> 
> Sending in a patch to make it work with Python 3, seems to work fine with 
> Python 2 too. Basically uses the "new" syntax for print and exceptions and 
> opens a few files in binary mode as they are not plaintext.

Great!, maybe it's better to just commit what I have, so that you can
commit this patch directly. 

> Albert
> 
> > 

Regards, 
-- 
Carlos Garcia Campos
PGP key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x523E6462
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/poppler/attachments/20110913/5ec058c9/attachment-0001.pgp>