[poppler] [RFC] Extend regtest framework to track performance

[Adam Reichold]

Hello again,

as discussed in the code modernization thread, if we are going to make
performance-orient changes, we need a simple way to track functional and
performance regressions.

The attached patch tries to extend the existing Python-based regtest
framework to measure run time and memory usage to spot significant
performance changes in the sense of relative deviations w.r.t. to these
two parameters. It also collects the sums of both which might be used as
"ball park" numbers to compare the performance effect of changes over
document collections.

The patch runs the measured commands repeatedly including warm-up
iterations and collects statistics from these runs. The measurement
results are stored as JSON documents with the actual program output of
e.g. pdftotext or pdftoppm being discarded.

To implement the check for relative deviations, it abuses the checksum
comparison method and hence checksums are still computed for the JSON
documents even though they are actually unnecessary. It is also limited
to Unix-like operating systems (due to the use of the wait3 syscall to
determine resource usage similar to the time command).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: regtest-performance.patch
Type: text/x-patch
Size: 27189 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/poppler/attachments/20151230/bd4eb25d/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freedesktop.org/archives/poppler/attachments/20151230/bd4eb25d/attachment-0001.sig>