[cairo] [PATCH] Make utilites buildable on modern Linux distro (RHEL-7).

[Bryce Harrington]

On Fri, Mar 06, 2015 at 09:25:04AM +0100, Matěj Cepl wrote:
> On 2015-03-06, 01:41 GMT, Bryce Harrington wrote:
> > Personally for cairo-trace, I think csi-trace and csi-bind should be
> > used from the master trunk instead of having their own copies.  They're
> > already starting to diverge a bit.  Then cairo-trace could become a
> > data-only repository and not need a build system at all, which would be
> > nice.  I think a new filetype would need to be established for the
> > compressed traces, though.
> 
> Actually I am all for making cairo-trace clean data-only 
> repository. Given its size, I don’t it should be made into 
> proprer distro package (I feel compassion for all those mirrors 
> who carry this beast), so I think it should be used just on 
> as-needed basis by some testing scripts.

In fact if it were data-only, we could further consider having it broken
up into several smaller or more focused repositories.  Like you say,
it's rather a beast as it is currently; would be nice to have a subset
that runs quicker but still gives reasonable coverage.

I'm not certain exactly how people have been using the traces to
understand the use case, but simply looking at the variety of traces I
get the impression that webbrowser performance optimization has been a
important mission.  Which makes me wonder if an interesting breakdown
would be per-browser: So one repository with a full set of traces taken
for firefox, another repository with the equivalent traces but taken
from webkit.  And so forth.

> Then it would be way 
> more simple if all executables were in the cairo proper (aren’t 
> they anyway? I see util/cairo-script/csi-bind.c and 
> util/cairo-script/csi-trace.c in the main repo).

If you do a diff you'll see they've diverged a bit.  csi-bind.c in the
main repo supports a few new cmdline options for example.  The
divergence for csi-trace.c looks a bit thornier but I'm sure it could be
sorted.

More importantly, in the cairo-trace repository, the traces can't be run
immediately since they're stored as compressed files.  The repository
includes a make command to uncompress them (just run 'make').  However I
don't think this is a great solution, because it assumes you want to
uncompress *all* of the traces, and this can gobble a LOT of disk space.
This may be less of an issue if the repository were to be broken up into
smaller ones, but as it stands it's a bit of a problem.

%.trace: %.lzma csi-bind
        lzma -cd $< | ./csi-bind > $@

At this point you could run the performance tests individually, but
there is also a perf/cairo-perf-trace.c tool in mainline to run them.
It looks for *.trace files and runs them all.  If we changed this to
run directly on lzma files (by doing the decompress and csi-bind steps
within cairo-perf-trace.c), then we can eliminate the Makefile from
cairo-traces entirely, and usage becomes much simpler.

As an additional note from the above make rule, after the compressed
file is uncompressed with lzma, it still has to be run through csi-bind
before it can be used.  So before this is done, it's in an intermediate
'unbound' format.  csi-bind optimizes the trace for the local cairo.
I think we should go ahead and define an actual file extension for this
intermediate file type, like *.traz or something.


> However, that is not the biggest problem I have. I was working 
> for a week on making a way how to run perf/* utilities during 
> the ordinary RHEL QA testing. In the end I have crashed and 
> burned on the fact that perf/* utilities apparently could be run 
> only from the build tree and they cannot be installed (and I am 
> not a C programmer by any stretch of imagination).  Would you 
> (or anybody) have an idea how to make perf/* utils (especially 
> cairo-perf-micro, but cairo-perf-diff and cairo-analyse-trace 
> would be lovely as well) into something which could run even 
> after making install from /usr/bin? There seems to be some steps 
> towards it in the Debian package patches 
> (https://sources.debian.net/src/cairo/1.14.0-2.1/debian/patches\
> /01_build_perf_utils.patch/) but it would be lovely if we got 
> all this to the main cairo repository.

Debian builds those tools and packages them into the 'cairo-perf-utils'
package.  See debian/cairo-perf-utils.*

Probably the right way to do this on our end would be to add a
--enable-perf-utils option to configure, which would cause those
binaries to be built and installed.  There's probably other utilities in
the tree worth making available for installation.
 
However, there is a big implication if we do this, that we're promising
to keep the command line interface and functional behaviors of these
utilities reasonably stable henceforth.

> In the end I have just written script which downloads all 
> required packages, builds cairo, and then runs tests from the 
> build tree, but I really don’t like that.
> 
> Any ideas how to proceed?

Most of the utilities appear to support either command line arguments or
an environment variable to tell them where to look for *.trace files, or
where to store test output.  The documentation may be sketchy though, so
you'd need to examine the source code to see how the util can be run.

So, a first step would be to improve perf/README with this information.
man pages would be even better.

Making the scripts themselves use a consistent env variable or command
line switch for where the *.trace and output directories are would also
be a simple way to improve the situation here.

Bryce