XDG Default Applications specification proposal

[elektra at markus-raab.org]

Dear XDG list,

short summary as the email got very long: I think the success of the
D-Bus specification blinded about the fact that not in every situation a
specification is the best solution. For configuration, a protocol that
describes how to get or set a configuration value is not feasible. We
should agree on a path to configuration value and let accessing the
configuration to be an implementation detail to something like Elektra.

Details below :-)

Am 22.07.20 um 17:40 schrieb Simon McVittie:
> On Wed, 22 Jul 2020 at 14:39:22 +0200, elektra at markus-raab.org wrote:
>> Would you also see dbus for notification as implementation detail?
> 
> Yes; for Notifications, dbus *is* an implementation detail, but D-Bus
> is not. (I'm being very careful with case combination and spelling here.)
> 
> The Notifications API requires using D-Bus, which is a protocol
> that anyone can implement if they want to, like HTTP or X11. There

Yes, for notification writing a specification about the protocol is an
obvious step. And you were successful with it!

> is a somewhat formal specification of how to find a D-Bus socket on a
> freedesktop system, how to authenticate to it, how to exchange messages
> (the "wire protocol"), and what the well-known system and session buses
> are for. Notifications builds on top of that to define some specific
> D-Bus messages.

Elektra has many specifications:

 - its API: https://doc.libelektra.org/api/
 - a >300 pages book capturing the meta-specification
   https://book.libelektra.org
 - many grammars that specify different configuration file formats
 - different protocols and technologies that ensure that applications do
   not overwrite each others configurations
 - different ways to locate the configuration files on different
   operating systems (like XDG Base Directory Spec)

After this long journey, I can say with high confidence that it is not
feasible to specify how to get or set a configuration value.

What is feasible and done by Elektra is to specify an abstraction,
comparable to a virtual file system (which allows mounting the
configuration files) and an API that allows access (like the
open/read/write/close API for filesystems).

> The closest equivalent for a file-based interface, like Desktop Entry and
> mimeapps.list, would be documenting what the files are, how to find them

Trying to specify this would be as difficult as trying to specify how an
virtual file system works and leave it "up to the implementation" to
find the file systems on the hard discs and then to locate files there.

This might work for reading files (e.g. grub does it) but if you also
want to modify something, you will get very lost very soon.

> (which is done by referencing the Base Directory spec rather than directly,

The Base Directory spec does not help at all to *change* configuration
in an interoperable way. And the problems of environment variables were
already discussed before.

> just like the way Notifications references D-Bus rather than specifying
> how to find the session bus itself), and what their contents mean.
> 
> The D-Bus protocol specification isn't perfect (clarifications welcome
> via merge requests!), but it exists, and is sufficiently useful that
> there are at least four independent implementations that I know of.

If these four implementations can talk to each other it seems to be good
enough.

> [...]>
> If there is a single implementation that is optimal for *everyone*, then,
> great, everyone can use that. X11 historically had many implementations,
> but now Xorg is the dominant implementation (possibly the only one used
> in practice).

Xorg made a clever system where different drivers have enough freedom to
do what they need to do. At Elektra we made a plugin system, where
plugins can do what they need to do (e.g. locate and read/write
configuration files). I think Elektra found a sweet spot, which could be
good enough for anyone, as it can be very fast (because it has an mmap
cache) and is also very flexible (as nearly all functionality is in
plugins).

> However, we don't know ahead of time whether a single implementation
> is going to be suitable for everyone. dbus was intended to be optimal

Of course we do not know without trying it.

> for everyone's D-Bus needs, which we now know is a really difficult
> thing to achieve. Because dbus tries to be library-stack-agnostic,
> desktop-agnostic, suitable for everyone's unusual requirements and
> fully backward-compatible, it ended up being equally *unsuitable* for
> everyone, and difficult to maintain and improve, to the extent that its
> maintainers[1] frequently recommend using a different implementation,

You also have to keep in mind that dbus was the first implementation, so
obviously others were able to learn from that.

> like GDBus or sd-bus, that can be made more suitable for certain
> use-cases by not being bound by the same constraints. For example,
> GDBus requires dispatching the GLib main loop, and sd-bus does not allow
> sharing connections between threads. Both of those are fine choices for
> those particular implementations, which improve their suitability for
> some uses, but neither of those design choices would have been accepted
> in dbus, because our constraints are not the same as theirs.
> That's fine, and not a bug: put different constraints in and you get
> different implementations out.

Yes. In Elektra there are many such situations and thus also many plugins.

>> Or to put it bluntly: After DBus not much XDG *technology* happened
>> anymore and XDG degraded to a discussion club, often about environment
>> variables. And this despite the fact that environment variables are not
>> well-suited and configuration settings are for a free desktop at least
>> as important as inter-process communication.
> 
> I think you might be overestimating how much power "XDG" actually

Quite a few things from XDG are heavily adopted. And there is no
alternative to XDG anyway?

> has. There is no formal XDG organization, there is nobody who is
> specifically paid to maintain or work on XDG, and, crucially, there is
> nobody who can compel the maintainers of distros, desktop environments,
> applications etc. to follow a particular specification by saying it's
> an XDG standard.

This is an advantage. I am very much for this freedom.

I do not want Elektra to have only one implementation. And if you find
something better than Elektra, please take it!

> This is not the ISO, the IETF, or even POSIX.

> An XDG specification has value if the relevant components (in this case
> terminal emulators, and desktop libraries like GLib and Qt/kdelibs[2])
> adopt it. If they don't adopt it, then the specification has little or no
> value, because other relevant components can't rely on it.

Yes.

> In the case of a default-terminal specification, the only way it can
> become useful is if the maintainers of major libraries that launch
> .desktop files, for which the elephants in the room are GLib and
> Qt/kdelibs, either merge an implementation that has been contributed to
> them or write their own (which might reuse lower-level libraries, or not,
> depending on implementation choices). If those maintainers decide the
> specification isn't suitable - rightly or wrongly, it doesn't actually
> matter! - then that makes it a lot less useful, perhaps even entirely
> useless, because applications can't rely on it.
> 
> In my experience, a specification being tied to a specific implementation
> is something that desktop environment implementors tend to be suspicious
> of. It's more likely to be widely adopted if it's implementation-neutral,
> like the proposal from Thayne that started this particular thread (which
> is just spec wording, and doesn't come with an implementation at all),
> or the older and more narrowly-targeted proposal from Vladimir Kudrya
> (which comes with a reference implementation, but implementors aren't
> required to use it, and I expect most wouldn't.).
> 
> To put it even more bluntly: it doesn't matter how much you like a
> specification, and it doesn't matter how much I like it either. What
> matters most is whether the maintainers of GLib and Qt/kdelibs (and
> other implementations, but mostly those two) like it enough to merge an
> implementation of it.

Shouldn't it be the goal that the maintainers are also involved in the
discussion and we find the best solution according to the requirements
we have?

> If there are particular aspects of a specification (file formats,
> locations, semantics, whatever) that make life hard for you as an
> implementor, then you're very welcome to suggest changes that make it
> more straightforward,

Yes, I suggest such changes. I do not think that the filesystem is a
suitable abstraction to specify configuration in the same way as the
filesystem is not the suitable abstraction for databases. E.g. SQL is
one suitable abstraction to access relational databases. I think Elektra
API a suitable abstraction for configuration.

> without losing the properties that it was designed
> to have; but if they are at the level of "use this framework and it
> will do everything you need" rather than at the level of interoperating
> file formats,

Interoperating file formats are not the solution for everything, in
particular they are not well-suited for interoperability of
configuration because:

1. it is impossible to agree on where the files are, as different
   operating systems have fundamentally different requirements.
2. specifying configuration file formats with the properties we want
   (order-preserving, comments-preserving, ...) is way too hard and in
   the end also a matter of taste (like filesystems).

> then I suspect that maintainers will tend to view that
> in the same way as, for example, a Qt developer saying "everyone should
> standardize on QtCore because it's great", or a GNOME developer saying
> "everyone should standardize on GLib because it's great".

I hope we are past that discussions ;) I think it is commonly agreed
that for desktops the needs are too diverse to have a single implementation.

>> Without Elektra becoming a
>> dependency, Elektra will fail on the requirement to be available in
>> every distro.
> 
> I could say: without GLib becoming a dependency, GLib will fail on the
> requirement to be available in every distro. Does that mean Qt/KDE developers
> should be willing to accept XDG specifications that are written in terms of
> linking to GLib, and using GSettings or GVfs or something?
> (This seems unlikely to happen; Qt/KDE developers have had the opportunity
> to depend on GLib for literally decades, and have not done so, despite the
> amount of code they could stop needing to maintain by doing so.)

On my system, KDE applications link against GLib. But this is not where
I wanted to go with the discussion nor I have expertise to recommend
anything about that.

I would, however, not recommend to KDE to use GSettings.

> Conversely, a KDE developer could say the same about Qt or kdelibs.
> Does that mean GNOME developers should be willing to accept specifications
> that are written in terms of linking to Qt, and using QML or whatever?
> (This seems equally unlikely to happen, for similar reasons.)
> 
> Or substitute any other pair of implementations
> (MATE? LXQT? Enlightenment? WindowMaker?) for GLib/GNOME and Qt/KDE.
> 
> From an abstract, high-level point of view, the vaguely-defined
> "freedesktop platform" would seem to benefit a lot from everyone agreeing
> to use one particular library stack (or for that matter, one particular
> UI toolkit, OS distribution or kernel), and yet this has still not
> happened.

It cannot happen for similar reasons why we will not be able to find a
common configuration file format. The requirements and expectations
differ widely.

But it is no technical problem to write backends for the configuration
systems of all these library stacks, so that they all can start share
their configuration. Missing is only the will to go this direction.

> I think we can conclude that for the people who do the work,
> the (perceived or real) benefits of continuing to use their preferred
> stack outweigh the benefits of standardizing on one implementation.
> Not everyone has the same requirements, or the same priorities, or even
> the same preferences.

Of course.

> Perhaps Elektra is amazing and everyone should be using it, but if that's
> the case, it should be on its own merits, not because the nonexistent XDG
> cabal says so.

Of course. But as long XDG defines that configuration should be in that
and that file in that syntax it makes the idea of a global configuration
more and more harder. Nobody has a chance of implementing all these
configuration file formats out there.

So at least open-minded people like XDG should stop that and start
thinking about a more abstract view of configuration, where
configuration values can be accessed via paths (or keys). To make
something like this happen, we need to agree on something like Elektra.

> I'm sure we've been here before with other frameworks, and I'm sure
> we'll come back here again. Some get widely adopted as a dependency
> (polkit, libarchive, NetworkManager, udev, arguably systemd); some are
> used for a while, then later dropped when design or implementation issues
> become clear, or just because they stopped being maintained (ConsoleKit,
> Telepathy, gconf, ESD, ARTS); some never really make it at all (APOC,
> Galago, Ytstenut); some are designed to be cross-desktop but in practice
> are only adopted by a subset (dconf, tracker, folks);

dconf is *not* designed to be cross-desktop, it is deeply connected to
glib (it uses GVariant).

> and some are widely adopted but deeply controversial (systemd, again).

I'll better not start this discussion ;)

>     smcv

Thank you very much for this productive discussion, I think it helped to
frame Elektra better for XDG.

> [1] I am one of dbus' maintainers.

Disclaimer: for me it is Elektra :-)

> [2] I don't know which layer in that stack is responsible for implementing
>     .desktop files, but whichever layer has their equivalent of GLib's
>     GDesktopAppInfo, that would probably be the right place.


best regards,
-- 
Markus Raab          http://www.complang.tuwien.ac.at/raab/
TU Wien                   markus.raab at complang.tuwien.ac.at
Compilers and Languages          Phone:      +43 2619 21073
Argentinierstr. 8, 1040 Wien, Austria           DVR 0005886