[poppler] RFC: patch: parallel testing

Adam Reichold adamreichold at myopera.com
Sat Dec 29 06:22:08 PST 2012 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Thomas,

It is not a very sophisticated solution, but I think the attached
patch should ensure useful messages for interleaved output of several
worker threads.

Best regards, Adam.

Am 29.12.2012 10:32, schrieb Thomas Freitag:
> Hi Adam!
> 
> I used yesterday the first time Your patch for regtesting: Awfully,
> congratulations. You probably already encountered my comment on bug
> 50992. One minor thing: Unfortunately the protocol is not really
> readable anymore, because each thread i.e. first outputs
> 
> Testing '/media/thomas/HD-PCTU3/PDF Suite/01_ump_a_2009.pdf' using
>  splash backend (9/1137):
> 
> and when it finishes i.e.
> 
> PASS
> 
> Because of the worker threads it's now nearly impossible to match 
> the test itself with the result of the test. Can You change it in 
> that way that this is outputted together? Should be quite easy....
> 
> Cheers, Thomas
> 
> Am 06.12.2012 19:02, schrieb Adam Reichold: Hello again,
> 
> I reformatted the patch hopefully solving the whitespace issues. 
> (It does apply and run for me, but there are still some whitespace 
> warnings.)
> 
> The use of recursive locks is probably not necessary in 
> "TestRun.py" and "TestReferences.py", but since the performance 
> impact is probably negligible in this case, I thought allowing for 
> recursion would help to keep the code more accessible. In 
> "Printer.py" there is at least one recursion from
> "printout_update" into "printout".
> 
> Best regards, Adam.
> 
> Am 06.12.2012 18:19, schrieb Adam Reichold:
>>>> Hello Carlos,
>>>> 
>>>> Am 06.12.2012 18:11, schrieb Carlos Garcia Campos:
>>>>> Adam Reichold <adamreichold at myopera.com> writes:
>>>>>> Hello again,
>>>>>> 
>>>>>> As suggested by William Bader, I added the option to use 
>>>>>> the logical CPU count as the number of worker threads 
>>>>>> automatically.
>>>>> Thanks for the patch, any improvement in the regression 
>>>>> test framework is more than welcome, even more if it's to 
>>>>> improve the performance. What poppler have you sued to 
>>>>> create the patches? They don't apply for me here using 
>>>>> poppler from current git master.
>>>> Yes, it is about performance and I am hesitant about 
>>>> introducing such complexity into the test driver which is 
>>>> supposed to test the correctness of program itself, but 
>>>> faster regression testing probably means more regression 
>>>> testing... :-)
>>>> 
>>>> I am not sure how to correctly format the patch. I built on 
>>>> the current master but it seems Git thinks that the lines 
>>>> where I just changed the indentation are whitespace errors.
>>>> I am not sure how to correct this as changes in whitespace
>>>> are obviously important for Python scripts... I'll try to 
>>>> research this, but any advice would be welcome.
>>>> 
>>>> Best regards, Adam.
>>>> 
>>>>>> Regards, Adam.
>>>>>> 
>>>>>> Am 04.12.2012 12:28, schrieb Adam Reichold:
>>>>>>> Hello Thomas,
>>>>>>> 
>>>>>>> Am 04.12.2012 08:41, schrieb Thomas Freitag:
>>>>>>>> Am 04.12.2012 07:45, schrieb Adam Reichold: Hello 
>>>>>>>> everyone,
>>>>>>>> 
>>>>>>>> I currently try to get myself acquainted with 
>>>>>>>> Poppler's regression testing framework. Because my 
>>>>>>>> system has a rather low single-threaded performance, 
>>>>>>>> I tried to implement parallel testing using Python's 
>>>>>>>> Queue class.
>>>>>>>> 
>>>>>>>> Even though poppler-regtest currently uses two 
>>>>>>>> processes per test file, rendering even and odd
>>>>>>>> pages respectively, the test files themselves are
>>>>>>>> still handled sequentially and both process are
>>>>>>>> joined for each test file. This will yield suboptimal
>>>>>>>> system utilization even for a small three-core system
>>>>>>>> like mine.
>>>>>>>> 
>>>>>>>> Using the "-t/--threads N" option in the patched 
>>>>>>>> poppler-regtest will spawn N worker threads that 
>>>>>>>> handle all tests they can get from a single queue
>>>>>>>> for all known tests, allowing to heavily utilize
>>>>>>>> also large system if using a large set of test cases.
>>>>>>>> But even for my three-core system, this brought down
>>>>>>>> the time to create references for the complete test
>>>>>>>> suite using the Splash backend from 4,5 hours to
>>>>>>>> 2,75 hours.
>>>>>>>>> What do You mean with "patched poppler-regtest"? 
>>>>>>>>> There was no attachment, or do I miss something?
>>>>>>> No, this on the process level in the test driver and 
>>>>>>> independent of the your multi-threading work. There
>>>>>>> was a patch called "parallel_testing.patch" attached,
>>>>>>> as least my mail client tells me so.
>>>>>>> 
>>>>>>> I'll just try again...
>>>>>>> 
>>>>>>> Best regards, Adam.
>>>>>>> 
>>>>>>>>> If You talk about my multi-threaded testcase, 
>>>>>>>>> please be aware, that it is still experimental, 
>>>>>>>>> even if quite close. Here the result of my last 
>>>>>>>>> regression test last sunday evening: Total 1133 
>>>>>>>>> tests 1114 tests passed (98.32%) 17 tests failed 
>>>>>>>>> (1.50%): /media/thomas/HD-PCTU3/PDF 
>>>>>>>>> Suite/Algorithmics - The Spirit of Computing, 3rd 
>>>>>>>>> Ed.pdf (splash), /media/thomas/HD-PCTU3/PDF 
>>>>>>>>> Suite/Essentials of English Grammar - 
>>>>>>>>> www.ielts4u.blogfa.com.pdf ... 2 tests crashed 
>>>>>>>>> (0.18%): /media/thomas/HD-PCTU3/PDF 
>>>>>>>>> Suite/bug157090.pdf (splash), 
>>>>>>>>> /media/thomas/HD-PCTU3/PDF Suite/sinatr4c.f5.pdf 
>>>>>>>>> (splash) I'll first have a look at the crashes
>>>>>>>>> next weekend, then I'll continue with looking at
>>>>>>>>> the failed tests.... Cheers, Thomas
>>>>>>>> 
>>>>>>>> IMHO, the necessary changes seem quite small 
>>>>>>>> especially since a lot of them are connected to 
>>>>>>>> indentation handling. What are your thoughts on the 
>>>>>>>> utility and implementation of this?
>>>>>>>> 
>>>>>>>> Best regards, Adam.
>>>>>>>>> 
>>>>>>>>> _______________________________________________ 
>>>>>>>>> poppler mailing list poppler at lists.freedesktop.org
>>>>>>>>> 
>>>>>>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> 
_______________________________________________ poppler
>>>>>>>> mailing list poppler at lists.freedesktop.org 
>>>>>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 
_______________________________________________ poppler
>>>>>>>> mailing list poppler at lists.freedesktop.org 
>>>>>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>>>
>>>>
>>>>>>>>
>>>>>>>> 
_______________________________________________ poppler mailing
>>>> list poppler at lists.freedesktop.org 
>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>>> 
>>>> 
>> 
>> 
>> _______________________________________________ poppler mailing 
>> list poppler at lists.freedesktop.org 
>> http://lists.freedesktop.org/mailman/listinfo/poppler
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iQEcBAEBAgAGBQJQ3vyQAAoJEPSSjE3STU34/9kH/iOG+N6WLsZKKKYeKtxEOtda
TMT1s6SsiWCEQIHIZNMoJvpHAO75mDXU3M+/QXuYre6Bhn+1+PlsVI/S74wus9HI
8jJC0sDOBYvM3PrEng+kjMsff/iOH6hsMrPbL/R/qAB4qjdCQSeAvloYu4TaSeB4
6dyqoZ05qxUt5UMEtVqDQLGHMllL2fwOMy3//elpzCiDArR/okv51p9r+qiEbQAe
7a+xpRplFSbQs0AvNqlE1GFFGVRusckzH6aXLng//DiCbJ+8iyGqZ5ALqZyk/HeH
fRb4e+68siIRf+NvpUktNJCfmvjqfF47yaW+e3JAxTm848sHtmEqHqeEyT/wL98=
=CKSU
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix_regtest_print.patch
Type: text/x-patch
Size: 3539 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/poppler/attachments/20121229/6975aa21/attachment.bin>

More information about the poppler mailing list