[poppler] RFC: patch: parallel testing

[Adam Reichold]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Carlos,

Am 06.12.2012 18:11, schrieb Carlos Garcia Campos:
> Adam Reichold <adamreichold at myopera.com> writes:
> 
>> Hello again,
>> 
>> As suggested by William Bader, I added the option to use the 
>> logical CPU count as the number of worker threads automatically.
> 
> Thanks for the patch, any improvement in the regression test 
> framework is more than welcome, even more if it's to improve the 
> performance. What poppler have you sued to create the patches?
> They don't apply for me here using poppler from current git
> master.

Yes, it is about performance and I am hesitant about introducing such
complexity into the test driver which is supposed to test the
correctness of program itself, but faster regression testing probably
means more regression testing... :-)

I am not sure how to correctly format the patch. I built on the
current master but it seems Git thinks that the lines where I just
changed the indentation are whitespace errors. I am not sure how to
correct this as changes in whitespace are obviously important for
Python scripts... I'll try to research this, but any advice would be
welcome.

Best regards, Adam.

>> Regards, Adam.
>> 
>> Am 04.12.2012 12:28, schrieb Adam Reichold:
>>> Hello Thomas,
>>> 
>>> Am 04.12.2012 08:41, schrieb Thomas Freitag:
>>>> Am 04.12.2012 07:45, schrieb Adam Reichold: Hello everyone,
>>>> 
>>>> I currently try to get myself acquainted with Poppler's 
>>>> regression testing framework. Because my system has a rather 
>>>> low single-threaded performance, I tried to implement 
>>>> parallel testing using Python's Queue class.
>>>> 
>>>> Even though poppler-regtest currently uses two processes per 
>>>> test file, rendering even and odd pages respectively, the 
>>>> test files themselves are still handled sequentially and
>>>> both process are joined for each test file. This will yield 
>>>> suboptimal system utilization even for a small three-core 
>>>> system like mine.
>>>> 
>>>> Using the "-t/--threads N" option in the patched 
>>>> poppler-regtest will spawn N worker threads that handle all 
>>>> tests they can get from a single queue for all known tests, 
>>>> allowing to heavily utilize also large system if using a 
>>>> large set of test cases. But even for my three-core system, 
>>>> this brought down the time to create references for the 
>>>> complete test suite using the Splash backend from 4,5 hours 
>>>> to 2,75 hours.
>>>>> What do You mean with "patched poppler-regtest"? There was 
>>>>> no attachment, or do I miss something?
>>> 
>>> No, this on the process level in the test driver and 
>>> independent of the your multi-threading work. There was a
>>> patch called "parallel_testing.patch" attached, as least my
>>> mail client tells me so.
>>> 
>>> I'll just try again...
>>> 
>>> Best regards, Adam.
>>> 
>>>>> If You talk about my multi-threaded testcase, please be 
>>>>> aware, that it is still experimental, even if quite close. 
>>>>> Here the result of my last regression test last sunday 
>>>>> evening:
>>>> 
>>>>> Total 1133 tests 1114 tests passed (98.32%) 17 tests
>>>>> failed (1.50%): /media/thomas/HD-PCTU3/PDF
>>>>> Suite/Algorithmics - The Spirit of Computing, 3rd Ed.pdf
>>>>> (splash), /media/thomas/HD-PCTU3/PDF Suite/Essentials of
>>>>> English Grammar - www.ielts4u.blogfa.com.pdf ... 2 tests
>>>>> crashed (0.18%): /media/thomas/HD-PCTU3/PDF
>>>>> Suite/bug157090.pdf (splash), /media/thomas/HD-PCTU3/PDF
>>>>> Suite/sinatr4c.f5.pdf (splash) I'll first have a look at
>>>>> the crashes next weekend, then I'll continue with looking
>>>>> at the failed tests....
>>>> 
>>>>> Cheers, Thomas
>>>> 
>>>> 
>>>> IMHO, the necessary changes seem quite small especially
>>>> since a lot of them are connected to indentation handling.
>>>> What are your thoughts on the utility and implementation of
>>>> this?
>>>> 
>>>> Best regards, Adam.
>>>>> 
>>>>> 
>>>>> _______________________________________________ poppler 
>>>>> mailing list poppler at lists.freedesktop.org 
>>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________ poppler 
>>>> mailing list poppler at lists.freedesktop.org 
>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________ poppler 
>>>> mailing list poppler at lists.freedesktop.org 
>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
> 
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iQEcBAEBAgAGBQJQwNO/AAoJEPSSjE3STU34m1AIAJJkb9hOAYUrEV5lv4ZMiuOk
zfaDCJvFlWZyrDb/20rjDiwD4FrZjhymKujuCaKFGnNmClZCobUCEX6KrSJjlFhK
UPASIIcfeQs2t23kP0qigQTQB4tqMSXRBWpoHgezeoc7WQM4s6F9nCapuG42WAGX
EQaJqFCf4FBHmg49p+iiXOz64DBcszZowNwgAAe9LZ7hAStBa2hRsHxzu2kCT8+s
gyfZymIuTI8ai4FUJazLqotBmvHgKFhrSGhVuxiMNyEaa5evz6sZQI7NUvzgXRbc
z5kC8fnCkc3ci85+MkhWQGQzeaQYQ95D+bimyuWCcZ2vF/mb6cw10t7Hens8hNk=
=vPP3
-----END PGP SIGNATURE-----