[poppler] RFC: patch: parallel testing

Thu Dec 6 10:02:24 PST 2012

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello again,

I reformatted the patch hopefully solving the whitespace issues. (It
does apply and run for me, but there are still some whitespace warnings.)

The use of recursive locks is probably not necessary in "TestRun.py"
and "TestReferences.py", but since the performance impact is probably
negligible in this case, I thought allowing for recursion would help
to keep the code more accessible. In "Printer.py" there is at least
one recursion from "printout_update" into "printout".

Best regards, Adam.

Am 06.12.2012 18:19, schrieb Adam Reichold:
> Hello Carlos,
> 
> Am 06.12.2012 18:11, schrieb Carlos Garcia Campos:
>> Adam Reichold <adamreichold at myopera.com> writes:
> 
>>> Hello again,
>>> 
>>> As suggested by William Bader, I added the option to use the 
>>> logical CPU count as the number of worker threads
>>> automatically.
> 
>> Thanks for the patch, any improvement in the regression test 
>> framework is more than welcome, even more if it's to improve the
>>  performance. What poppler have you sued to create the patches? 
>> They don't apply for me here using poppler from current git 
>> master.
> 
> Yes, it is about performance and I am hesitant about introducing
> such complexity into the test driver which is supposed to test the 
> correctness of program itself, but faster regression testing
> probably means more regression testing... :-)
> 
> I am not sure how to correctly format the patch. I built on the 
> current master but it seems Git thinks that the lines where I just 
> changed the indentation are whitespace errors. I am not sure how
> to correct this as changes in whitespace are obviously important
> for Python scripts... I'll try to research this, but any advice
> would be welcome.
> 
> Best regards, Adam.
> 
>>> Regards, Adam.
>>> 
>>> Am 04.12.2012 12:28, schrieb Adam Reichold:
>>>> Hello Thomas,
>>>> 
>>>> Am 04.12.2012 08:41, schrieb Thomas Freitag:
>>>>> Am 04.12.2012 07:45, schrieb Adam Reichold: Hello
>>>>> everyone,
>>>>> 
>>>>> I currently try to get myself acquainted with Poppler's 
>>>>> regression testing framework. Because my system has a
>>>>> rather low single-threaded performance, I tried to
>>>>> implement parallel testing using Python's Queue class.
>>>>> 
>>>>> Even though poppler-regtest currently uses two processes
>>>>> per test file, rendering even and odd pages respectively,
>>>>> the test files themselves are still handled sequentially
>>>>> and both process are joined for each test file. This will
>>>>> yield suboptimal system utilization even for a small
>>>>> three-core system like mine.
>>>>> 
>>>>> Using the "-t/--threads N" option in the patched 
>>>>> poppler-regtest will spawn N worker threads that handle all
>>>>>  tests they can get from a single queue for all known
>>>>> tests, allowing to heavily utilize also large system if
>>>>> using a large set of test cases. But even for my three-core
>>>>> system, this brought down the time to create references for
>>>>> the complete test suite using the Splash backend from 4,5
>>>>> hours to 2,75 hours.
>>>>>> What do You mean with "patched poppler-regtest"? There
>>>>>> was no attachment, or do I miss something?
>>>> 
>>>> No, this on the process level in the test driver and 
>>>> independent of the your multi-threading work. There was a 
>>>> patch called "parallel_testing.patch" attached, as least my 
>>>> mail client tells me so.
>>>> 
>>>> I'll just try again...
>>>> 
>>>> Best regards, Adam.
>>>> 
>>>>>> If You talk about my multi-threaded testcase, please be 
>>>>>> aware, that it is still experimental, even if quite
>>>>>> close. Here the result of my last regression test last
>>>>>> sunday evening:
>>>>> 
>>>>>> Total 1133 tests 1114 tests passed (98.32%) 17 tests 
>>>>>> failed (1.50%): /media/thomas/HD-PCTU3/PDF 
>>>>>> Suite/Algorithmics - The Spirit of Computing, 3rd Ed.pdf 
>>>>>> (splash), /media/thomas/HD-PCTU3/PDF Suite/Essentials of 
>>>>>> English Grammar - www.ielts4u.blogfa.com.pdf ... 2 tests 
>>>>>> crashed (0.18%): /media/thomas/HD-PCTU3/PDF 
>>>>>> Suite/bug157090.pdf (splash), /media/thomas/HD-PCTU3/PDF 
>>>>>> Suite/sinatr4c.f5.pdf (splash) I'll first have a look at 
>>>>>> the crashes next weekend, then I'll continue with
>>>>>> looking at the failed tests....
>>>>> 
>>>>>> Cheers, Thomas
>>>>> 
>>>>> 
>>>>> IMHO, the necessary changes seem quite small especially 
>>>>> since a lot of them are connected to indentation handling. 
>>>>> What are your thoughts on the utility and implementation
>>>>> of this?
>>>>> 
>>>>> Best regards, Adam.
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________ poppler 
>>>>>> mailing list poppler at lists.freedesktop.org 
>>>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________ poppler 
>>>>> mailing list poppler at lists.freedesktop.org 
>>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________ poppler 
>>>>> mailing list poppler at lists.freedesktop.org 
>>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
> 
> 
> _______________________________________________ poppler mailing
> list poppler at lists.freedesktop.org 
> http://lists.freedesktop.org/mailman/listinfo/poppler
> 
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iQEcBAEBAgAGBQJQwN2wAAoJEPSSjE3STU34FwsIAJRvIKHcb/pZS4pRp6FsoLof
vdIg9O1cmFKp/9AroV2iYw4ScnlGQaa4TauN8SnDFs3vZC432I8/UQnp/P7nkSld
4HdCXXa6LFpxQQa+K4klU3IfH/ZEKmjLpc7njl23OStxk948IgzJZMTkCeYvl/F6
qQQFgYhlTpPSdnGQB5M41JMERNMeGlwLgBJuPFjZjeG6EGHeI1pUBY7qpPLAR6tC
m+wjAG6ZBdOn9+Nkc61GWGv1gNmGWiSi1/IRy1MsBY50/aSg/6iMM870ccGovYHa
sDXEFOs3Rp0nmWEpBh5UjtZpzC/A/CM/UCe49KiBERrkVvfkgTlpdekpfdzlCM0=
=LN68
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: parallel_testing.patch
Type: text/x-patch
Size: 13036 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/poppler/attachments/20121206/81af8ed4/attachment-0001.bin>