pkg-config and cross-compiling redux

[Rüdiger Kuhlmann]

Hi!

>--[Owen Taylor]--<otaylor at redhat.com>

First of: I neither care about how you interpolate the $HOST nor how
you actually call the environment variable to use.

>  - When pkg-config is invoked as <host>-pkg-config, have it extract
>    the <host> part and substitute it for occurrences of $HOST in
>    $PKG_CONFIG_LIBDIR. 
>  - Add a $PKG_CONFIG_CROSS_LIBDIR variable to allow configuration
>    of cross compilation as above while not disturbing native 
>    builds.

My proposal wouldn't interpolate anything into $PKG_CONFIG_LIBDIR, it only
would use $PKG_CONFIG_CROSS_LIBDIR and interpolate it there if
cross-compilation was detected. What use would it have to do that on
$PKG_CONFIG_LIBDIR? The desired path for cross-compilation might differ
more than just by a $HOST directory from the one for native compiling.

>  - Add $PKG_CONFIG_DESTDIR ($PKG_CONFIG_CROSS_DESTDIR) variables
>    with subtitution as above that would set a $_destdir variable.
>    This would allow .pc files to be written so they work either
>    on the host or target system.

My proposal wouldn't make changes to the .pc files; it would just overwrite
the "prefix" directory with $PKGCONFIG_XPREFIX (which does
$HOST-interpolation as well) unless it's set to the empty string. Which is
what I prefer because it wouldn't require changes to the existing .pc files.
In particular, note that Windows absolute paths cannot be relative paths, so
$_destdir wouldn't work at all.

| (This discussion might be better on xdg-list really)

Sure. But it seams no one commented on it yet. :-(

| > This patch seems to be adding a lot of assumptions about the directory 
| > > setup for cross-compilation that aren't necessarily universal. 
| > Which? They just assume that you install pkg-config and the host's in the same 
| > place, which is reasonable. And, as you see, you can override them with 
| > environment variable to match your layout. 
| You patch assumes an installation with prefix /usr/<host>/, which
| while not unheard of, is likely uncommon ...

Well, /usr is the prefix used in pretty much all distributions.

| most people don't want to install custom builds into /usr. Yes, you can
| override that with an envvar, but I don't see the advantage of that over
| just setting PKG_CONFIG_LIBDIR for the usual case.

Well, that way you can make packages that install in the default place,
which is a good place to chose if you're distributing packages. If you don't
install into those places, you need to set $PKG_CONFIG_CROSS_LIBDIR, just
like you have to set $PKG_CONFIG_LIBDIR if you install packages with .pc
files somewhere else (... and you have to copy all existing .pc files you
want to reuse ...), and just like you have to set $PATH if you install
something otherwise than in the default place. That's why I think it's the
best place for it.

| > > Frankly, setting PKG_CONFIG_LIBDIR doesn't seem any harder to me  
| > > than creating a symlink to <host>-pkg-config. And much easier to 
| > > understand. 
| > No, it's not easier to understand. Have all your build's .pc files in a wierd 
| > place - just set PKG_CONFIG_LIBDIR. Have all yout cross-compile .pc files in a 
| > wierd place - just set PKG_CONFIG_X_LIBDIR. This way you adapt them to your 
| > layout, while otherwise you need to set them according to what you 
| > cross-compile just now. My approach means: configure your layout (with sane 
| > defaults given). Your approach: set an env var each time you cross-compile. 
| > See the difference? Your approach is basically manual intervention into the 
| > build process. Which is what you want to avoid. 
| If you are cross-compiling to half-a-dozen different architectures regularly
| with a uniform directory layout, then I guess that might be true. How many
| people do that? If we can keep the general use case
| (one cross-compilation target at a time) simpler, I don't think asking
| such people to have some shell scripts to set up their environment
| is unreasonable.

You're right that not many people do cross-compile for more than one
platform (though I've cross-compiled for three different platforms
altogether, though with a completely different setup for one of them).
However, Doing it this way allows you to distribute packages of your
cross-compiling tools that just work everywhere because then there's a
reasonable default. A single user might not install more than one
cross-compilation environment, but if he would, they'd all be the same
layout.

| Here's a possible compromise: instead of adding another environment
| variable with an unclear interaction with PKG_CONFIG_LIBDIR, why don't
| we simply allow variable substitutions of $PLATFORM into PKG_CONFIG_LIBDIR. (I
| suggest that token because it's supported by RPATH in the linux dynamic loader,
| but $HOST might make more sense) I think that's cleaner.

I don't think so. There is no unclear interaction with $PKG_CONFIG_LIBDIR.
Unless you call pkg-config as <host>-pkg-config, it will behave in exactly
the same way as before. And if you call it that way, you declare that way
that you want to have cross-compiling, so using $PKG_CONFIG_CROSS_LIBDIR
instead is in fact the cleanest way. You'd have $PKG_CONFIG_LIBDIR for
native builds and $PKG_CONFIG_CROSS_LIBDIR for cross compiles. You then
could have both in totally different locations (like a common case would be
to have $PKG_CONFIG_LIBDIR in /usr/lib/pkg-config and
$PKG_CONFIG_CROSS_LIBDIR in $HOME/lib/%s/pkg-config.

One could actually think whether one could support a
$PKG_CONFIG_<host>_LIBDIR variable for the specific host given as well,
though that might be not used that often (since, as you said, people don't
cross-compile for that many targets at the same time.

| > > - Add support for building pkg-config with --target=; the only effect 
| > >   of which would be to install the binary as <target>-pkg-config 
| > >   (this might be close to working already?) 
| > Then you have basically one binary for each possible host platform. In my case 
| > you have a binary that fits all, and you just need to add a symlink. 
| And figure out a printf-formatted envvar and get that into your environment...

Well, your proposal to interpolate $HOST may be better (though you' have to
get an ugly $ into the env var), but you need to figure that out as well.
And if it's in the man page that wouldn't be a problem.

| Since you need a separate binary for all the other tools,
| I don't see having one for pkg-config as much of an issue. As soon
| as you have to do anything (create a symlink even), that's the majority
| of the conceptual complexity. But sure, if we can avoid it in some
| simple way, that's great.

That's right in so far as the complexity is the same. But why would you want
to install a new binary if you could have one global for all? A symlink is
less work and uses less disc space. You could actually add a --host option
to pkg-config and cause the .m4 to call it the right way, it's just that
that would be more complex than changing it to call <host>-pkg-config as
just using a different autoconf macro does the trick, and <host>-pkg-config
allows the user to setup a wrapper if for any reason more wierd tricks need
to be done for his target platform.

| > > One could also argue that one should try to address the big problem - 
| > > sharing .pc files between cross compilation on the host and  
| > > native compilation on the build - first and then perhaps the  
| > > right solution little problem (finding the right .pc files for  
| > > cross compilation) will fall out of that. 
| > Well, that's what the prefix override solves. Of course you can argue that you 
| > could do that with a seperate copy of <host>-pkg-config as well. 
| I don't see how the prefix override really solves things, because you
| the different-from-install path on the host system, not on the build system. 

I can't really parse that. I guess I was thinking of being able to re-use
.pc files generated natively on the host system and then re-used on the
build system, which it does solve. For the vice versa case, you'd have to
specify the host's install paths with --prefix and then do a staged install
to where you want to use it on the build system (though that will cause
trouble for Windows paths - how do they do staged install on Windows
anyway?).

| Also, overriding the "prefix" variable is likely a bad idea, because the
| prefix variable is just a convention that we happened to start using
| in the config files. (The basic reason it's there, truth be told, is
| because it avoids really nasty shell substitution untangling in the
| configure file.) It also doesn't work if the user explicitely specified
| --libdir on their configure line, because then libdir=@libdir@
| will look like libdir=/usr/lib not libdir=${prefix}/lib.

Hmm... that's right, though it's a really good idea to use stuff relative to
$prefix.  Requiring to do so for cross-compilation to work would solve it by
excluding the cases where it would break.

| It seems that a solution here would need to involve DESTDIR in some fashion so
| that the correct host paths get put into the .pc files.

Well, that would require updating all .pc files to that scheme.

Would it be possible to detect whether $DESTDIR has been seen, and if not,
override $prefix? That would mean that existing .pc files could be reused
as long as they use $prefix.

Did you know that pkg-config will guess a $prefix when compiled for the W32
platform depending on the location of the .pc file?

| We could add a new special internal variable (not allowed to set it in the .pc
| file) $_destdir say, so that in gtk+-2.0.pc we would write:

Well, why $_destdir and not $destdir or $DESTDIR? (okay, my $PKG_CONFIG_XPREFIX
might have been a bad choice as well, seems $PKG_CONFIG_DESTDIR would be the
right name).

| Then you would set 
|  PKG_CONFIG_DESTDIR=/mnt/build/$PLATFORM

Or not setting anything at all and use the stuff in /usr/$PLATFORM *g*.

-- 
         100 DM =  51  € 13 ¢.
         100  € = 195 DM 58 pf.
  mailto:ruediger at ruediger-kuhlmann.de
    http://www.ruediger-kuhlmann.de/