[poppler] RFC: patch: parallel testing

Sat Dec 29 01:32:12 PST 2012

Hi Adam!

I used yesterday the first time Your patch for regtesting: Awfully, 
congratulations. You probably already encountered my comment on bug 
50992. One minor thing: Unfortunately the protocol is not really 
readable anymore, because each thread i.e. first outputs

Testing '/media/thomas/HD-PCTU3/PDF Suite/01_ump_a_2009.pdf' using 
splash backend (9/1137):

and when it finishes i.e.

PASS

Because of the worker threads it's now nearly impossible to match the 
test itself with the result of the test. Can You change it in that way 
that this is outputted together? Should be quite easy....

Cheers,
Thomas

Am 06.12.2012 19:02, schrieb Adam Reichold:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hello again,
>
> I reformatted the patch hopefully solving the whitespace issues. (It
> does apply and run for me, but there are still some whitespace warnings.)
>
> The use of recursive locks is probably not necessary in "TestRun.py"
> and "TestReferences.py", but since the performance impact is probably
> negligible in this case, I thought allowing for recursion would help
> to keep the code more accessible. In "Printer.py" there is at least
> one recursion from "printout_update" into "printout".
>
> Best regards, Adam.
>
> Am 06.12.2012 18:19, schrieb Adam Reichold:
>> Hello Carlos,
>>
>> Am 06.12.2012 18:11, schrieb Carlos Garcia Campos:
>>> Adam Reichold <adamreichold at myopera.com> writes:
>>>> Hello again,
>>>>
>>>> As suggested by William Bader, I added the option to use the
>>>> logical CPU count as the number of worker threads
>>>> automatically.
>>> Thanks for the patch, any improvement in the regression test
>>> framework is more than welcome, even more if it's to improve the
>>>   performance. What poppler have you sued to create the patches?
>>> They don't apply for me here using poppler from current git
>>> master.
>> Yes, it is about performance and I am hesitant about introducing
>> such complexity into the test driver which is supposed to test the
>> correctness of program itself, but faster regression testing
>> probably means more regression testing... :-)
>>
>> I am not sure how to correctly format the patch. I built on the
>> current master but it seems Git thinks that the lines where I just
>> changed the indentation are whitespace errors. I am not sure how
>> to correct this as changes in whitespace are obviously important
>> for Python scripts... I'll try to research this, but any advice
>> would be welcome.
>>
>> Best regards, Adam.
>>
>>>> Regards, Adam.
>>>>
>>>> Am 04.12.2012 12:28, schrieb Adam Reichold:
>>>>> Hello Thomas,
>>>>>
>>>>> Am 04.12.2012 08:41, schrieb Thomas Freitag:
>>>>>> Am 04.12.2012 07:45, schrieb Adam Reichold: Hello
>>>>>> everyone,
>>>>>>
>>>>>> I currently try to get myself acquainted with Poppler's
>>>>>> regression testing framework. Because my system has a
>>>>>> rather low single-threaded performance, I tried to
>>>>>> implement parallel testing using Python's Queue class.
>>>>>>
>>>>>> Even though poppler-regtest currently uses two processes
>>>>>> per test file, rendering even and odd pages respectively,
>>>>>> the test files themselves are still handled sequentially
>>>>>> and both process are joined for each test file. This will
>>>>>> yield suboptimal system utilization even for a small
>>>>>> three-core system like mine.
>>>>>>
>>>>>> Using the "-t/--threads N" option in the patched
>>>>>> poppler-regtest will spawn N worker threads that handle all
>>>>>>   tests they can get from a single queue for all known
>>>>>> tests, allowing to heavily utilize also large system if
>>>>>> using a large set of test cases. But even for my three-core
>>>>>> system, this brought down the time to create references for
>>>>>> the complete test suite using the Splash backend from 4,5
>>>>>> hours to 2,75 hours.
>>>>>>> What do You mean with "patched poppler-regtest"? There
>>>>>>> was no attachment, or do I miss something?
>>>>> No, this on the process level in the test driver and
>>>>> independent of the your multi-threading work. There was a
>>>>> patch called "parallel_testing.patch" attached, as least my
>>>>> mail client tells me so.
>>>>>
>>>>> I'll just try again...
>>>>>
>>>>> Best regards, Adam.
>>>>>
>>>>>>> If You talk about my multi-threaded testcase, please be
>>>>>>> aware, that it is still experimental, even if quite
>>>>>>> close. Here the result of my last regression test last
>>>>>>> sunday evening:
>>>>>>> Total 1133 tests 1114 tests passed (98.32%) 17 tests
>>>>>>> failed (1.50%): /media/thomas/HD-PCTU3/PDF
>>>>>>> Suite/Algorithmics - The Spirit of Computing, 3rd Ed.pdf
>>>>>>> (splash), /media/thomas/HD-PCTU3/PDF Suite/Essentials of
>>>>>>> English Grammar - www.ielts4u.blogfa.com.pdf ... 2 tests
>>>>>>> crashed (0.18%): /media/thomas/HD-PCTU3/PDF
>>>>>>> Suite/bug157090.pdf (splash), /media/thomas/HD-PCTU3/PDF
>>>>>>> Suite/sinatr4c.f5.pdf (splash) I'll first have a look at
>>>>>>> the crashes next weekend, then I'll continue with
>>>>>>> looking at the failed tests....
>>>>>>> Cheers, Thomas
>>>>>>
>>>>>> IMHO, the necessary changes seem quite small especially
>>>>>> since a lot of them are connected to indentation handling.
>>>>>> What are your thoughts on the utility and implementation
>>>>>> of this?
>>>>>>
>>>>>> Best regards, Adam.
>>>>>>>
>>>>>>> _______________________________________________ poppler
>>>>>>> mailing list poppler at lists.freedesktop.org
>>>>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>>>>>
>>>>>>
>>>>>> _______________________________________________ poppler
>>>>>> mailing list poppler at lists.freedesktop.org
>>>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________ poppler
>>>>>> mailing list poppler at lists.freedesktop.org
>>>>>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>
>> _______________________________________________ poppler mailing
>> list poppler at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/poppler
>>
>>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.19 (GNU/Linux)
>
> iQEcBAEBAgAGBQJQwN2wAAoJEPSSjE3STU34FwsIAJRvIKHcb/pZS4pRp6FsoLof
> vdIg9O1cmFKp/9AroV2iYw4ScnlGQaa4TauN8SnDFs3vZC432I8/UQnp/P7nkSld
> 4HdCXXa6LFpxQQa+K4klU3IfH/ZEKmjLpc7njl23OStxk948IgzJZMTkCeYvl/F6
> qQQFgYhlTpPSdnGQB5M41JMERNMeGlwLgBJuPFjZjeG6EGHeI1pUBY7qpPLAR6tC
> m+wjAG6ZBdOn9+Nkc61GWGv1gNmGWiSi1/IRy1MsBY50/aSg/6iMM870ccGovYHa
> sDXEFOs3Rp0nmWEpBh5UjtZpzC/A/CM/UCe49KiBERrkVvfkgTlpdekpfdzlCM0=
> =LN68
> -----END PGP SIGNATURE-----
>
>
> _______________________________________________
> poppler mailing list
> poppler at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/poppler

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/poppler/attachments/20121229/3f87689b/attachment-0001.html>