[poppler] Poppler reg tests

[Carlos Garcia Campos]

Excerpts from Albert Astals Cid's message of mar sep 13 14:13:37 +0200 2011:
> A Dimarts, 13 de setembre de 2011, Carlos Garcia Campos vàreu escriure:
> > Excerpts from Albert Astals Cid's message of mar sep 13 12:27:27 +0200 2011:
> > > A Dilluns, 12 de setembre de 2011, Carlos Garcia Campos vàreu escriure:
> > > > Hi all,
> > > 
> > > Hi
> > > 
> > > > as you all probably know, our current regression test suite are some
> > > > custom shell scripts and a lot of pdf files that Albert has
> > > > collected
> > > > for some years. During the last few days I have been working on
> > > > improving the regression test, converting Albert's scripts into a
> > > > small python program to allow everybody to run their own tests. We
> > > > are
> > > > not going to upload pdfs to the repo, so we still depend on Albert
> > > > to make sure a change doesn't actually introduce regressions.
> > > > 
> > > > SNIP
> > > > 
> > > > There are still things to do, like adding an option to provide
> > > > documents to skip,
> > > 
> > > Yeah, the documents to skip is important, i have a few files here that
> > > cause infinite loops because of errors in things we do not control
> > > (like openjpeg) and i need to be able of ignoring those.
> > 
> > For now, as a workaround you could rename the files to skip as something
> > like pdf_document_i_want_to_skip.pdf.skip, since only .pdf files are
> > used.
> 
> Poor man solution, but yeah, would work

It's a workaround not a real fix, I'll implement proper skipped list soon.

> > > It would be good to have some sense of how the test is progressing, the
> > > problem at the moment is that files are tested in "random" order, so can
> > > not use the current file to know how much is left. It would be good
> > > either to sort the files or to get a "1 of 1000" text to know how much
> > > is left
> > 
> > Yes, it happened to me yesterday too, it would take a bit longer, but
> > I guess it's better to get information about the tests first, sort it,
> > and then iterate it to create the refs and run the tests.
> 
> Yeah, time "lost" time should be really small compared to real running time

Sure, I'll do it.

> > 
> > > I am not sure the "Test run in X seconds" works, see here
> > > 
> > > $ time ~/devel/poppler/regtest/poppler-regtest --utils-
> > > dir=/home/tsdgeos/devel/poppler/build-old/utils/ --backends=splash
> > > run-tests --refs-dir=./refs --out-dir=./out .
> > > Testing './alumnes_normativa.pdf' using splash backend: PASS
> > > Total 1 tests
> > > 1 tests passed (100.00%)
> > > Tests run in 0 seconds
> > > 
> > > real    0m1.101s
> > > user    0m1.690s
> > > sys     0m0.097s
> > > 
> > > That 0 seconds seem wrong either to the 1.1 real or 1.6 user seconds
> > 
> > The timer only counts the time to create-refs and run-test, no the
> > whole program, it shouldn't be much different, though. I assumed that
> > in most of the cases it will take more than once, and if it didn't,
> > the time spent doesn't really matter, so it doesn't give
> > milliseconds. Try with a test that takes a bit longer.
> 
> Ok, giving it more work reported correct times.

Great.

> > 
> > > > html backend support, and I'm sure there are a lot of bugs
> > > > too. So, feel free to comment, or even better provide patches :-)
> > > 
> > > Sending in a patch to make it work with Python 3, seems to work fine
> > > with
> > > Python 2 too. Basically uses the "new" syntax for print and exceptions
> > > and opens a few files in binary mode as they are not plaintext.
> > 
> > Great!, maybe it's better to just commit what I have, so that you can
> > commit this patch directly.
> 
> I'm a bit hesitant to commiting this before we branch for 0.18. If you want to 
> commit it now i can create the branch for 0.18.x today so you can commit it to 
> master.

Note that this is intended to be used from a git clone, it's not
thought to be included in tarballs, so it shouldn't affect any
release.

> Albert
> 
> > 
> > > Albert
> > 
> > Regards,

Regards, 
-- 
Carlos Garcia Campos
PGP key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x523E6462
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/poppler/attachments/20110913/911058fa/attachment.pgp>