App Bundles

[Brian J. Tarricone]

On 08/27/2009 02:19 AM, André Gillibert wrote:
> "Brian J. Tarricone" <bjt23 at cornell.edu> wrote:
> 
>> LD_LIBRARY_PATH isn't so useful since it rarely contains the entire
>> library search path, and is often empty.  On my system, /etc/ld.so.conf
>> contains 10 paths in it, and that usually doesn't include the default
>> /lib and /usr/lib (at least it need not include those).
>>
> I don't see your point.

I'm not talking about the runtime dynamic linker finding shared
libraries that were linked at compile time to the executable.  (Though,
admittedly, this is *another* problem we'd need to solve if we want to
allow bundles to include their own private copies of libraries.)

I'm talking about an application specifically enumerating files in
certain paths to look for shared objects (e.g. plugins) to dlopen().

Currently most applications that use plugins will require that plugins
are installed to the same location that the app is installed.  Some do
things like have environment variables (e.g., GTK_PATH) that allow you
to modify the search path, but this is non-standard and would need to be
set separately for every app that supports it.  Apps that don't support
it are left out in the cold, or would need to be patched to add their
own non-standard env var.

Take for example Audacious, the music player.  It looks for audio output
plugins in $libdir/audacious/Output, or, in my case,
/usr/lib64/audacious/Output.  It looks there because that particular
path is hard-coded into the audacious binary.  It does not, for example,
look in /lib64/audacious/Output, or in /usr/local/lib/audacious/Output,
or, perhaps more usefully, in
/opt/plugin-i-want-to-test/lib/audacious/Output.  I cannot change this
path without recompiling.  I cannot make it look in multiple places[1].

Let's take an analogous example that doesn't involve libraries.  When my
session manager starts up, it runs autostart applications.  It does not
have a hard-coded place (or set of places) to look[2].  It looks in a
colon-separated list of directories in the XDG_CONFIG_DIRS environment
variable, and in XDG_CONFIG_HOME.  I can tell it to search whatever
system (or non-system) directories I wish.  Since I have Xfce installed
to /opt/xfce4-git, I add /opt/xfce4-git/etc/xdg to XDG_CONFIG_DIRS and
everything works just fine.

Now let's apply this to our bundle (obviously this is all hypothetical).
 Before launching the bundle, my file manager will look at
XDG_CONFIG_DIRS, and add the path to the bundle's Resources
subdirectory.  The application in that bundle doesn't need to know a
thing about bundles.  It just follows the XDG basedir spec and happily
finds its files.

But what about plugins?  In order to load its plugins, it needs to know
where they are.  They're in some subdirectory of the bundle, say
Resources/lib/plugins.  But how does it find them?  Well, with how
things are currently done, it can't.  At least, not without
modifications to how it's built, and how it resolves paths.

>> My use case here is in app with plugins.
> For plugins, applications should simply build a path based on their bundle path they get from argv[0] and call dlopen on their own.

Unacceptable.  This means that every application that uses plugins will
need to be aware of whether or not it's in a bundle, or at least will
need to be aware of a dynamic prefix.  We already have the ability to do
this -- see autopackage's relocatable stuff (which don't even require
autopackage).  But who actually uses it?  I can think of very few offhand.

If people are serious about this bundle thing, it needs to work without
needing major modifications to applications.  Or, rather, it needs to
work within a framework not specific to the bundle.  XDG_CONFIG_DIRS and
XDG_DATA_DIRS clearly buy me several advantages as an app author (and
the apps's user) that have nothing to do with making relocatable
bundles.  Something like XDG_PLUGIN_DIRS would as well, and it would be
a benefit to all applications, whether packaged as a bundle, or packaged
in the 'traditional' manner.

> IMO, env vars are dangerous for bundled applications, since they're inherited by child processes.
> The bundled application may be an application launcher itself!
>
>> If we had XDG_PLUGIN_DIRS, our bundle launcher could dynamically prepend a path
>> under our bundle directory to XDG_PLUGIN_DIRS before launching the
>> plugin executable.
> 
> And how would an application recognize which plugins are compatible with itself in the mess of dirs you would get when chaining bundled applications with fork/exec.

If the app doesn't already have a means to recognize plugins that aren't
compatible (via some internal versioning system), then it's broken.
Look at gstreamer, for example, which uses a versioned directory name
which changes when the plugin interface breaks ABI.  Xfce's old
configuration system used an ABI version number embedded in the .so that
would get checked by the configuration manager after dlopen()ing it.

If an app is installing its plugin files directly into a shared
namespace (e.g. /usr/lib instead of /usr/lib/foo-app), then it deserves
to break anyway.

If Foo-app looks for its plugins in $XDG_PLUGIN_DIRS/foo-app, and
Bar-app looks for its plugins in $XDG_PLUGIN_DIRS/bar-app, then there
can never be any conflict, even if both of them are set up as bundles
and one launches the other with their "private" bundle path in
XDG_PLUGIN_DIRS.

> Rule of the thumb:
> 1) If you want a setting to be inherited by exec'ed processes, use an env var.
> 2) If you don't want so, use a parameter (argv[0] being one of them).

That's nice in theory, but when you're trying to convince hundreds of
app authors to do extra work to support your pet packaging format that
no one uses, it sounds like a hard sell.

In this case I'm suggesting that it *doesn't matter* if XDG_PLUGIN_DIRS
has extraneous entries in it.  If apps are written properly, there's
just no problem.  If they're not written properly, then that's a bug,
and they'll probably have issues when installed in the traditional
manner too.

> So, the plugin dir should be pointed by a a parameter.
> Since it should be a subdir of the bundled application dir and you've already defined a way to get this dir (argv[0]), the application has everything it needs.

No, it doesn't have everything it needs.  It may have all the
*information* it needs, but it doesn't know to do anything with it
without extra work.  Even if that extra work is minimal, you still have
to convince the app author that doing that work is worthwhile.  Or you
have to go out and provide patches for hundreds of applications, and
shepherd these patches through each app's submission and review process.

Listen... I personally (for my own use) don't care if we have a bundle
system or not.  But I think it could be immensely useful for users who
don't want to touch a command line and just want to quickly try out a
piece of software that their distro hasn't packaged.  It's also a much
more logical and friendly flow to see something on a website, and
download it from that website and try it immediately and simply.

So please understand that my perspective here is from an application
author and desktop environment maintainer.  I'd be happy to write a
bundle launcher implementation for Thunar and xfdesktop, or have them
use one that could be provided in GIO/GVfs.  The launcher implementation
side is a small target: get Nautilus and Konqueror to implement bundle
launchers and you could conceivably say you're done on that side.

But then you need actual bundles to launch, and unless you make it dirt
simple and compelling for app authors to make their apps relocatable, it
just isn't going to happen.[3]  Don't take my word for it: look to
autopackage's relocation efforts and how well that took off.

	-brian

[1] Well, it probably also has a per-user plugin directory in $HOME, but
that's also hard-coded and doesn't address the problem here.

[2] Of course it has a fallback default search path if XDG_CONFIG_DIRS
isn't defined, but... not really the point.

[3] The holy grail would be... "drop this autoconf macro in $top_srcdir,
add FOO_CREATE_BUNDLE to configure.ac, run autoreconf, and type 'make
bundle'."  Bonus points if we can remove the first step in that.

Yes, I'm aware that something like XDG_PLUGIN_DIRS would require app
changes to support.  But my thesis here is that XDG_PLUGIN_DIRS is
*generically* useful, and something that app authors would likely adopt
without a bundle spec, similarly to how app authors adopted
XDG_CONFIG_DIRS and XDG_DATA_DIRS.