XDG_CONFIG_DIRS an /usr/local/etc/xdg

[Elsie Hupp]

> The way you describe it, it would be OK for any app to just parse the config of any other. That just feels wrong, because app A should have no business snooping in /etc/xdg/B/Brc. If app B wants to make such information available to others it should export it instead of requiring those to parse the file.

To be fair, this is one of the motivations behind Flatpak and Snap. If you don’t want one app snooping where it shouldn’t, then you make it technically unable to do so.

> On Sep 20, 2021, at 8:28 AM, Peter White <peter.white at posteo.net> wrote:
> 
> On Mon, Sep 20, 2021 at 10:20:05AM +0200, David Faure wrote:
>> On dimanche 19 septembre 2021 16:57:22 CEST Peter White wrote:
>>> On Sun, Sep 19, 2021 at 12:19:33PM +0200, David Faure wrote:
>>>> On jeudi 16 septembre 2021 18:48:41 CEST Peter White wrote:
>>>>> But, /etc should be off limits for software in /usr/local, right?
>>>> 
>>>> I don't think this assessment is correct.
>>>> 
>>>> For instance, I certainly expect KDE software installed in any prefix, to
>>>> respect the global settings in /etc/xdg/kdeglobals
>>> 
>>> Why would software need to read that file? 
>> 
>> Well, that's the point of config files, to be read by software, isn't it?
> 
> Well, duh, but not by *other* software. Why would any software have to
> read files it otherwise has no business with? kdeglobals is a file
> related directly to KDE. Plus, every app then also needs to *parse* that
> file. That sounds like an invitation for a whole lot of duplicated
> effort when, if those values are intended to be used, they should be
> available by other means, i.e. provide a library that abstracts that
> away.
> 
> The way you describe it, it would be OK for any app to just parse the
> config of any other. That just feels wrong, because app A should have no
> business snooping in /etc/xdg/B/Brc. If app B wants to make such
> information available to others it should export it instead of requiring those to
> parse the file.
> 
>>> Granted, I know virtually
>>> noting about KDE, but shouldn't there be a facility that makes those
>>> values available by other means, i.e. environment variables, or
>>> a settings daemon or whatever? 
>> 
>> You want one environment variable per setting in that file? That doesn't 
>> scale. A settings daemon might be what gnome does, it doesn't mean that all 
>> other free desktops want to have such an architecture. Surely reading files is 
>> still allowed in 2021?
> 
> Yes, it very much is, just not files of other applications. You seem to
> taking an "illegal" shortcut here, by expecting apps to do things that
> are otherwise frowned upon.
> 
>>> And BTW, shouldn't that be /usr/{,local/}share/kdeglobals [1]?
>> 
>> No, config files don't go to XDG_DATA_DIRS.
>> 
>>> 	> share/config/ ... A special case is "kdeglobals": this file is
>>> 	> 
>>> 	> 		    read by all KDE applications.
>>> 
>>> and then XDG_DATA_DIRS is the relevant env var which already has the
>>> correct default, as you point out below. Now, I don't know why "config"
>>> files would go anywhere other than ${PREFIX}/etc but that is apparently
>>> what KDE deems the right place.
>> 
>> The above documentation is really outdated, it says "kde 3" in many places,
>> it predates the XDG base dir spec (at least, its use by KDE).
> 
> Then provide a better reference? That's what my search came up with. ;)
> 
>>> Anyhow, if one really needs to make /etc/xdg/kdeglobals available to
>>> apps in /usr/local, then that is one special case that applies to KDE
>>> only
>> 
>> I don't believe so. As I said, everyone agrees that /usr is available to apps 
>> in /usr/local (since that's the default value for XDG_DATA_DIRS)
>> so why not do the same with config dirs?
> 
> And again, *you* talk about /usr and *I* talk about /etc. Compare that
> to PATH. There is no equivalent for /etc, because there doesn't need to
> be.
> 
>>> and that is the only case I can think of right now. 
>> 
>> There are lots of other files in /etc/xdg...
>> For instance /etc/xdg/user-dirs.conf which is not KDE specific at all.
> 
> And that also has nothing to do with the config of the app itself. It
> provides information about the environment.
> 
>>>> XDG_CONFIG_DIRS was modeled very much after XDG_DATA_DIRS,
>>>> where I would have tons of other examples like: apps in /usr/local or
>>>> anywhere else should still see /usr/share, for e.g. /usr/share/mime which
>>>> has the mimetype definitions.
>>> 
>>> That is not the same as /etc. The well known behaviour, prior to XDG,
>>> should not be broken for desktops and, as pointed out above, use the
>>> share/.. hierarchy then or whatever, since this seems very much like a
>>> KDE quirk to me and should not be baked into a standard that is supposed
>>> to agnostic of the environment.
>> 
>> Please stop saying this is a KDE quirk. It's the XDG base spec that defines 
>> /etc/xdg to be the default location for systemwide config files,
> 
> And I welcome that effort very much to reduce clutter in /etc. BTW, KDE
> should then also put kdeglobals in its own sub/base directory, i.e.
> /etc/xdg/kde/kdeglobals. That would make it more consistent with how
> XDG_CONFIG_HOME works, which XDG_CONFIG_DIRS is supposed to mirror
> system-wide.
> 
>>>> And yes, the intent is definitely that they should be read at runtime,
>>> 
>>> Yes, to find some global settings, maybe, but to find its *own* rc-file?
>> 
>> An app's own configuration is combination of the system wide defaults
>> (in /etc/xdg), its own installed defaults (in $PREFIX/etc/xdg),
> 
> Now we are getting somewhere...
> 
>> and the user's 
>> preferences (in $XDG_CONFIG_HOME). This assumes the XDG_CONFIG_DIRS is
>> set to contain the first two values, as is customary when using a custom 
>> prefix since the exact same thing has to be done with XDG_DATA_DIRS.
> 
> Why would XDG_CONFIG_DIRS need to contain ${PREFIX}/etc/xdg for that?
> The app pretty much already knows where it is supposed to find *its own*
> system-wide config. The location of which *should* be in
> ${PREFIX}/etc/xdg/<appname>/, yes, but one does not need to check
> XDG_CONFIG_DIRS for that at runtime. This is very much the reason for me
> starting this thread. App developers seem to think that they need to
> find *their own* configs this way, which invites ambiguity and probably
> unpredictable/unintended behaviour as I tried to point out with my worst
> case example.
> 
> It would make sense, though, to query XDG_CONFIG_DIRS at *compile time*,
> to see if the target prefers something different from the default
> /etc/xdg and just prepend ${PREFIX} to be more consistent with that.
> 
> And the more I think about it *and* seeing that no-one seems to ever set
> XDG_CONFIG_DIRS, which then defaults to /etc/xdg, this variable could
> just as well be renamed to the singular version XDG_CONFIG_DIR and only
> contain *one* value, either set explicitly or the current default. The
> whole reason for XDG base directory spec was reduction of clutter, after
> all. So why would you allow to have *multiple* base directories in the
> first place? There is only one XDG_CONFIG_HOME as well.
> 
> And please don't come talking about ../share again. ;) That is a
> different story entirely.
> 
>>>> so that [advanced] users can install things in custom prefixes and make
>>>> things work by setting a few env vars.
>>> 
>>> That is not something for "advanced" users, /usr/local is *the* very
>>> well known default for make.
>> 
>> I know, that's what I'm saying.
>> By custom I mean really custom (e.g. /opt or $HOME/install or whatever).
> 
> /opt has other implications and is usually reserved for apps one cannot
> or does not want to build locally, i.e. Firefox. One requirement is that
> *all* files need to reside there including the system-wide defaults.
> Now, if Firefox then needs *additional* information to better fit into
> its environment, *then* it can go and query XDG_CONFIG_DIR(S) but not
> for its *own* config. BTW, /etc/opt is missing from the default, because
> that is where additional configs for /opt are supposed to go, according
> to Filesystem Hierarchy Standard.
> 
>>> And the expectation is that everything
>>> needed to run software installed by 'make install' is available *inside*
>>> said prefix. 
>> 
>> Agreed. Hence /usr/local/etc/xdg.
> 
> But that breaks, as soon as there are multiple locations specified in
> XDG_CONFIG_DIRS, see my worst case.
> 
>>> Just look at the default PATH on any distro. It will contain
>>> /usr/local/bin and that will take precedence over /usr/bin. So one does
>>> not need to be "advanced" to run 'make && sudo make install' and then
>>> just run the locally built software by entering its binary's name
>>> without any further ado.
>> 
>> Agreed, hence my suggestion.
> 
> That is not the same. Again, PATH works on a *file* basis and the first
> match found wins, ignoring all others. If that were the case with
> XDG_CONFIG_DIRS, I would most certainly agree with you, but it works
> differently, so no. ;) Again, see my worst case.
> 
>>>> What it seems to me, is that /usr/local/etc/xdg should simply be added to
>>>> the default value for XDG_CONFIG_DIRS.
>>> 
>>> Again, that breaks prior art.
>> 
>> Bugfixing is about changing behaviour indeed, but for the better.
> 
> And yet it got worse. ;) There was no bug to fix in the first place. And
> XDG should stick to its premise and not invite clutter through the back
> door by means of multiple system wide base directories. There should
> really only be one: /etc/xdg by default. Mirroring that in any prefix is
> as easy as prepending it. Don't make things more complicated for the
> sake of it. ;)
> 
>>> Picture this: Software "foobar" installed in /usr/local finds *file*
>>> foobarrc in /usr/local/etc/xdg/foobar with *information*/setting A but
>>> not B in there, it will go on and read /etc/xdg/foobar/foobarrc and use
>>> setting B from there, if a distro provided version is also installed.
>> 
>> That's the point of layering config files, yes.
> 
> Yes, layering *user* config on top of the system-wide one but not
> layering one (up-to-date) version of the apps own config on top of an
> outdated one. There is a reason for etc being present in /usr/local. ;)
> 
>> You could add ~/.config/foobarrc to this sentence, it's a 3-layers view of the 
>> configuration.
> 
> Yes, if you want to get pedantic. I wanted to stick to the relevant
> stuff. ;)
> 
>>> In turn, running the distro version would also prefer setting A in
>>> /usr/local/etc/xdg/foobar/foobarrc despite there being A with a
>>> *different* value in /etc/xdg/foobar/foobarrc. Setting A in the most
>>> certainly made for a newer version of foobarrc in /usr/local could just
>>> as well lead to a DoS of the distro version of foobar, because at the
>>> time it was packaged the newer, now legal, value of A was illegal.
>>> And, since setting B was deprecated in the meantime the local version
>>> will also deny service.
>>> How is that for a worst case? ;)
>> 
>> You can cook up a similar scenario with /usr/share and /usr/local/share,
> 
> I invite you to try. ;)
> 
>> and yet I don't remember anyone ever having a problem with the distro version 
>> of an application picking up a wrong version of a file in /usr/local/share.
> 
> Because you don't seem to get the difference. That is, in fact, data
> meant to be shareable. And /usr/local/share is *meant* to override
> distro versions but also on a file basis. So, you are comparing apples
> to oranges here. I did notice a difference in the XDG spec compared to
> FHS, though. XDG also talks about information here, or at least it looks
> that way, where the FHS clearly only goes down to the file level. So the
> reason, no-one complained (so far) is that non-XDG software simply
> ignores all others after it has found a matching file in the most
> important directory. That is the *expected* behaviour, and quite
> frankly, I believe XDG should not change that by expecting apps to go on
> and read other files, because that makes deprecation unnecessarily hard.
> With FHS you would only need one file to mask the deprecated one. With
> the seemingly intended behaviour of XDG_DATA_DIRS there is no other way
> than having code that deals with that.
> 
> "Everthing is a file", remember? But XDG wants to get more granular and
> breaks many assumptions that come with that mantra. So at the very least
> the spec should elaborate on how to deal with that.
> 
>> I think the idea was that one would not install the same app into /usr and 
>> /usr/local. It was rather about making things work out of the box in both 
>> prefixes, for different apps.
> 
> Well, if that is the case, then it is a huge oversight, which I hinted
> at already.
> 
>>>> It's inconsistent that XDG_DATA_DIRS
>>>> defaults to /usr/local/share:/usr/share while XDG_CONFIG_DIRS defaults to
>>>> only /etc/xdg instead of /usr/local/etc/xdg:/etc/xdg
>>> 
>>> And there might be good reasons for that, as just outlined above, which
>>> is why I would very much like some input from the original authors on
>>> this topic.
>> 
>> Good luck. At least I'm pretty sure Waldo doesn't read this list.
> 
> Well, then what is it there for, if one cannot get information directly
> from the horse's mouth? But thanks for the hint. I will poke the authors
> to have a look at this thread then.
> 
>> But one thing is for sure: if there's a reason, it's not the fact that config 
>> files would be layered, because *that's the exact reason why XDG_CONFIG_DIRS is 
>> a set of dirs and not just one.*
> (emphasis added)
> 
> When in reality there is only one? Layering of config files is pretty
> much a very old and most certainly prior art. Look at how shells
> inititalize themselves. They do exactly that. User config trumps system
> wide config, but the order of reading the config files is from least
> important, i.e. /etc/bashrc, to most important, i.e. ~/.bashrc, simply
> overwriting values as they come, so the most important setting wins,
> because it is guaranteed to be the last read.
> 
>> And I remember that at the time, there was a 
>> strong use case for user groups sharing similar defaults, so the whole point 
>> was to set things up like /opt/admins/etc/xdg:/etc/xdg for some people
> 
> That should be /etc/opt/admins/etc/xdg:etc/xdg, just as an aside. ;)
> And since you only point to non-prefixed examples, I am more and more
> inclined to *not* set XDG_CONFIG_DIRS to include any prefix whatsoever.
> One can take XDG_CONFIG_DIRS as a template for ${PREFIX}, maybe, and at
> runtime just prepend ${PREFIX} when walking through XDG_CONFIG_DIRS.
> If that is acceptable for the spec I would appreciate an amendment to
> that effect. Even if one then sets:
> 
> XDG_CONFIG_DIRS=/usr/local/etc/xdg:/etc/xdg
> 
> the last match would win since the first would in reality point to
> /usr/local/usr/local/etc/xdg and result in a miss. For everything else,
> *aside* from its own config, use the prestine value of XDG_CONFIG_DIRS.
> 
> But I still fail to see why an app would even need to check that value
> to find its own config files.
> 
>> Because /etc/xdg isn't /usr/etc/xdg, the lack of /usr/local/etc/xdg
>> was overlooked.
> 
> Which I was suspecting from the beginning. If that is the case, then the
> spec needs amendments or at least some guidance on how to deal with it
> properly. First and foremost I would discourage app devs from abusing it
> to find their own config files, whose location they should know anyways.
> 
> But I would also encourage a rethink of the whole reading of files in
> /etc/xdg that are not directly related to an app. As I said, that is
> frowned upon where I come from. There should be a specified way of
> getting such information.
> At first glance $XDG_RUNTIME_DIR/xdg seems like a good fit.  But that
> should not be all. There must be a unified way of putting information
> there and, more importantly, reading it without having to resort to
> custom parsers for different origins of information. Think sysfs. So for
> kdeglobals that could be something like this:
> 
> 	> $ cat ${XDG_RUNTIME_DIR}/xdg/kde/kdeglobals/key_A
> 	> value_A
> 
>>> The main question still remains without a clear answer: Who or what
>>> should query XDG_CONFIG_DIRS and to what end? Regular software: should
>>> it really use it to find and then read its very own config file(s), the
>>> location of which being known at compile time anyways? Or is this for
>>> getting *other* information about the environment which said software
>>> has no other means of getting, like kdeglobals, for instance?
>> 
>> Regular software also uses libraries, which also have to find their own config 
>> files. "Known at compile time" leads to non-relocatable installations, which 
>> is rather restrictive for some use cases.
> 
> Name one. ;) Looking at the software I build locally to be more
> up-to-date, i.e. mpv, that is no problem. It even changes the docs to
> account for prefix. And relocatable is most certainly not what anyone
> expects from software installed by make or even the distro package
> manager. Everything has its proper place and there is simply no need for
> relocation after installation. That is the beauty of it all in a
> UNIX-ish world. Plus, there is /opt.
> 
>> Here's a definitive answer from the KDE side at least: *yes*, "regular 
>> software" uses XDG_CONFIG_DIRS and XDG_CONFIG_HOME to find all the config 
>> files it's looking for (both in the application code and in the library code).
> 
> Still incomplete: I get the environment stuff, but not the *its own
> config* part. ;) Because for that one only needs to make sure to
> put/expect it in the right location and save that path at
> compile/installation time. Everything else is just making things overly
> complex for no apparent gain or reason. Every LOC not written to deal
> with situations that are very unlikely to arise is a win. There are
> plenty possibilities to deal with those outside the code, or as Eric S.
> Raymond wrote in "The Art of UNIX programming": "Put the intelligence
> into the data (structure) not the code." I am paraphrasing, but that's
> the gist of it.
> 
> Just to be extra clear: I don't want to bash on XDG, I very much like
> the intention and the (partial) results that have come of it, i.e. less
> dotfile clutter in my $HOME or $XDG_DATA_HOME etc., I even bent my
> shell's config to use $XDG_CONFIG_HOME. But I am also sad that some
> things apparently have been overlooked. So there is still room for
> improvement and I hope the authors agree, seeing that the current
> version still starts with 0 (0.8).
> 
> 
> Best,
> PW