[poppler] RFC: patch: parallel testing

[Carlos Garcia Campos]

Adam Reichold <adamreichold at myopera.com> writes:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hello Carlos,
>
> Am 06.12.2012 18:11, schrieb Carlos Garcia Campos:
>> Adam Reichold <adamreichold at myopera.com> writes:
>> 
>>> Hello again,
>>> 
>>> As suggested by William Bader, I added the option to use the 
>>> logical CPU count as the number of worker threads automatically.
>> 
>> Thanks for the patch, any improvement in the regression test 
>> framework is more than welcome, even more if it's to improve the 
>> performance. What poppler have you sued to create the patches?
>> They don't apply for me here using poppler from current git
>> master.
>
> Yes, it is about performance and I am hesitant about introducing such
> complexity into the test driver which is supposed to test the
> correctness of program itself, but faster regression testing probably
> means more regression testing... :-)
>
> I am not sure how to correctly format the patch. I built on the
> current master but it seems Git thinks that the lines where I just
> changed the indentation are whitespace errors. I am not sure how to
> correct this as changes in whitespace are obviously important for
> Python scripts... I'll try to research this, but any advice would be
> welcome.

There's something weird with the patch, the reindented lines inside the
with blocks don't appear as modified in the patch. By the way, why using
RLock instead of just Lock?

> Best regards, Adam.
>
>>> Regards, Adam.
>>> 
>>> Am 04.12.2012 12:28, schrieb Adam Reichold:
>>>> Hello Thomas,
>>>> 
>>>> Am 04.12.2012 08:41, schrieb Thomas Freitag:
>>>>> Am 04.12.2012 07:45, schrieb Adam Reichold: Hello everyone,
>>>>> 
>>>>> I currently try to get myself acquainted with Poppler's 
>>>>> regression testing framework. Because my system has a rather 
>>>>> low single-threaded performance, I tried to implement 
>>>>> parallel testing using Python's Queue class.
>>>>> 
>>>>> Even though poppler-regtest currently uses two processes per 
>>>>> test file, rendering even and odd pages respectively, the 
>>>>> test files themselves are still handled sequentially and
>>>>> both process are joined for each test file. This will yield 
>>>>> suboptimal system utilization even for a small three-core 
>>>>> system like mine.
>>>>> 
>>>>> Using the "-t/--threads N" option in the patched 
>>>>> poppler-regtest will spawn N worker threads that handle all 
>>>>> tests they can get from a single queue for all known tests, 
>>>>> allowing to heavily utilize also large system if using a 
>>>>> large set of test cases. But even for my three-core system, 
>>>>> this brought down the time to create references for the 
>>>>> complete test suite using the Splash backend from 4,5 hours 
>>>>> to 2,75 hours.
>>>>>> What do You mean with "patched poppler-regtest"? There was 
>>>>>> no attachment, or do I miss something?
>>>> 
>>>> No, this on the process level in the test driver and 
>>>> independent of the your multi-threading work. There was a
>>>> patch called "parallel_testing.patch" attached, as least my
>>>> mail client tells me so.
>>>> 
>>>> I'll just try again...
>>>> 
>>>> Best regards, Adam.
>>>> 
>>>>>> If You talk about my multi-threaded testcase, please be 
>>>>>> aware, that it is still experimental, even if quite close. 
>>>>>> Here the result of my last regression test last sunday 
>>>>>> evening:
>>>>> 
>>>>>> Total 1133 tests 1114 tests passed (98.32%) 17 tests
>>>>>> failed (1.50%): /media/thomas/HD-PCTU3/PDF
>>>>>> Suite/Algorithmics - The Spirit of Computing, 3rd Ed.pdf
>>>>>> (splash), /media/thomas/HD-PCTU3/PDF Suite/Essentials of
>>>>>> English Grammar - www.ielts4u.blogfa.com.pdf ... 2 tests
>>>>>> crashed (0.18%): /media/thomas/HD-PCTU3/PDF
>>>>>> Suite/bug157090.pdf (splash), /media/thomas/HD-PCTU3/PDF
>>>>>> Suite/sinatr4c.f5.pdf (splash) I'll first have a look at
>>>>>> the crashes next weekend, then I'll continue with looking
>>>>>> at the failed tests....
>>>>> 
>>>>>> Cheers, Thomas
>>>>> 
>>>>> 
>>>>> IMHO, the necessary changes seem quite small especially
>>>>> since a lot of them are connected to indentation handling.
>>>>> What are your thoughts on the utility and implementation of
>>>>> this?
>>>>> 
>>>>> Best regards, Adam.
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________ poppler 
>>>>>> mailing list poppler at lists.freedesktop.org 
>>>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________ poppler 
>>>>> mailing list poppler at lists.freedesktop.org 
>>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________ poppler 
>>>>> mailing list poppler at lists.freedesktop.org 
>>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>> 
>> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.19 (GNU/Linux)
>
> iQEcBAEBAgAGBQJQwNO/AAoJEPSSjE3STU34m1AIAJJkb9hOAYUrEV5lv4ZMiuOk
> zfaDCJvFlWZyrDb/20rjDiwD4FrZjhymKujuCaKFGnNmClZCobUCEX6KrSJjlFhK
> UPASIIcfeQs2t23kP0qigQTQB4tqMSXRBWpoHgezeoc7WQM4s6F9nCapuG42WAGX
> EQaJqFCf4FBHmg49p+iiXOz64DBcszZowNwgAAe9LZ7hAStBa2hRsHxzu2kCT8+s
> gyfZymIuTI8ai4FUJazLqotBmvHgKFhrSGhVuxiMNyEaa5evz6sZQI7NUvzgXRbc
> z5kC8fnCkc3ci85+MkhWQGQzeaQYQ95D+bimyuWCcZ2vF/mb6cw10t7Hens8hNk=
> =vPP3
> -----END PGP SIGNATURE-----

-- 
Carlos Garcia Campos
PGP key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x523E6462
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/poppler/attachments/20121206/2ace7aa3/attachment.pgp>