DConf configuration system

[Avery Pennarun]

On Thu, Apr 07, 2005 at 03:35:23PM +0200, Joerg Barfurth wrote:

> Patrick Patterson wrote:
> >Well, we've (Net Integrations and the WvStreams Hackers) have already done 
> >a fair bit of work in this direction - along with the Elektra folk. 
> 
> Is there any place where I can see what is actually there (except the 
> source). I looked at the UniConf site repeatedly, but it appears to be 
> rather stale. In any case it appears to be much more targeted at 
> providing unified access to system or legacy configuration data than at 
> the kind of desktop configuration system that has been discussed here.

Our documentation is admittedly not the greatest; most of the users of
UniConf are inside our company, and they usually just ask someone in person. 
This is one of the traditional failings of commercial software development
that makes it hard to get projects accepted into the Open Source model.  On
the other hand, our code and our build system aren't disgusting, so at least
we skipped some of the other failings :)

There is semi-decent doxygen documentation.  Try this:

	http://open.nit.ca/wvstreams/docs/classUniIniGen.html

And the conference paper is the closest thing to a high-level architecture
description:

	http://open.nit.ca/uniconf.pdf

I admit this is crappy, but there you go.  We should fix it someday.

> >>>- Support for system defaults and some more selectable defaults for
> >>>different types of users, which stack one on top of other
> >
> >Yup - we've got a UniDefaultGen, and a UniPermGen.
> 
> I haven't found a clear description how multiple superimposed Gen things 
> interact; I have found nothing how this interacts with write access and 
> access control.

Patrick is slightly incorrect - UniDefaultGen is a misnamed generator that
doesn't apply here.  In fact, UniListGen is what you want instead.

The paper gives a brief introduction to this.  Essentially, you create a
list of generators in a stack (the most basic kind is ini:, so let's use
that in the example).  Each generator can have its own wrappers around it,
and UniPermGen is one kind of wrapper.  UniReadOnlyGen is another kind of
wrapper.  So you might do:

    list: {
	readonly:ini:/etc/system-overrides.ini
	ini:/home/apenwarr/.personal.ini
	readonly:ini:/etc/system-defaults.ini
    }

When trying to find a key, we look through each of the generators in order. 
system-overrides.ini will always win; personal.ini can be changed if the key
wasn't in system-overrides.ini; system-defaults.ini will be used if nothing
else matches.

Of course, a real system would use something more complex than the .ini
generator, because it's kind of evil to store *all* your options in a single
.ini file.  But you can, if you want.  (Among other things, there are gconf:
and dbus: generators you can use in place of ini:.)

> >>>- Undo/redo feature
> >
> >UniTransactionGen
> 
> Unod has at least two parts: One is 'transacted' multi-key operations. 
> Another is capturing the keys modified in a transaction and their prior 
> state. Do you put this all in your TransactionGen?

Patrick is jumping ahead a bit; UniTransactionGen is half of the solution.
The other half is an extension to the generator API that we haven't done
yet; luckily it's easy to do, but it's not done yet.  The extension is
simply a multiset() function that changes multiple keys at a time.  (Since
calling multiset() is inconvenient, the real purpose of UniTransactionGen is
to split that into a convenient start/set/set/set/commit sequence.)

If you have this, then you can have a generator or an application that
stores lists of changesets and can undo/redo them on demand.  Obviously
nobody has written such a generator/application yet, but since they haven't
written one for any other system either, merely being provably *able* to do
so ought to be enough for now.

> I am not conviced access to preexisting 'foreign' or 'native' 
> configuration systems is the primary use for pluggably backends in a 
> desktop configuration system. One reason is thst is hard to define a 
> meaningful and useful set of guaranteed semantics that any such backend 
> can provide. Usually one has to resort to a least common denominator 
> with limited utility (and this is the first impression I got on your 
> UniConfSemantics page).

This is true, in a way, and false, in a way.  UniConfSemantics is careful to
define two types of semantics: "full" (which is very powerful and not really
a lowest-common-denominator at all) and "limited" (which inherits the
semantic limitations of the backend).  In fact, full UniConfSemantics aren't
much of a limitation, but forcing your *backend* to comply with those
semantics (which, after all, require some flexibility) can be a bit of a
pain.

For example, gconf limits the possible names your keys can have.  Full
uniconf semantics do not.  The question is: does that matter in your
application?  If your application (say, a Gnome application) is only using
gconf-compatible key names anyhow, then a straight "limited" mapping makes
sense.  If your application is trying to store funky key names that gconf
can't, then you need a mapping that converts gconf-unfriendly keys to
gconf-friendly ones.  (For example, URL-encoding will fix up any keys gconf
doesn't like, without corrupting any ones that it does.)

The full-to-limited translation is often done as a wrapper plugin.  My
favourite example of this, while somewhat insane, is the UniConf "maildir"
plugin (don't try this at home!).  Maildirs are two-level hierarchies: level
one is the message-id, and level 2 is the list of headers and the body. 
That's obviously rather inflexible for storing arbitrary configuration key
layouts.  But I wrote a "mailwrap" plugin that converts arbitrary key/value
trees into maildir; it creates one email per key, and the subject line gets
the key string, while the body gets the value string.  You instantiate it
like this:

	mailwrap:maildir:/home/apenwarr/Maildir
	
What's my point?  Well, the short version of this is that it *is* meaningful
to provide a *complete* mapping of any limited-semantics backend into "full"
UniConf space.  We don't have to resort to a lowest-common-denominator
system.

> It really depends on what your understanding of 'transaction' and 
> 'integrity' is. I can't see how you can build anything that deserves the 
> name transaction on top of the semantics you require from the actual 
> storage backends.

Transaction semantics will indeed depend on the backends... depending how
you look at it.  If I use an .ini file, for example, I can't have
cross-application transactions without running a shared daemon.  That daemon
needs to support transactions in its own protocol, and needs to do atomic
updates to the .ini file occasionally (perhaps after each transaction).  If
it can do those two things, we have transactions all the way through the
system, even though we're using a cheesy .ini file that normally couldn't do
cross-process transactions.

(If your backend was a real database instead of a .ini file, you could have
transactions without sharing a UniConf daemon.)

> >UniConf talks via DBus :)
> 
> Do you have a documentation of your DBus API? I haven't even found one 
> for your C(++) interface.

The dbus uniconf plugin was written in the space of 2 hours the day before I
did a presentation at last year's desktopcon in Ottawa.  The speed at which
I could do this is a credit to the design of dbus, but that plugin hasn't
been used for anything and so there was no good reason to document it.  I
could write up some documentation about it in pretty short order, since it's
a nearly-direct translation of the UniConfGen API into dbus.  Get, set,
iterate, etc.  Nothing really awe-inspiring.  But then again, any dbus
protocol standard for a configuration system will have basically just those
things.

> I don't think you can attach access priviledges to keys or folders in 
> the namespace directly. In a stacked scenario they apply to instances of 
> a key in the various layers (e.g. I may be allowed to change my personal 
> preference, but not the group or site default). IME it is even 
> sufficient to apply access priviledges to entire layers. Any such access 
> control also has to correspond with other ways to modify the underlying 
> data. This fits the access control by layer model well. Protecting a key 
> is not much use, if the user can change it by manually editing a config 
> file. OTOH you don't want to store user preferences in a file that only 
> root can modify.

Whatever you want, some combination of UniListGen, UniPermGen, UniMountGen,
and UniReadOnlyGen can almost certainly do it.  I agree that my above
example with list/readonly is probably the most common case, and that's
great; it's also the easiest to explain.

> >Yes - UniMountGen does exactly what you want...
> 
> I still don't know what this is good for, if your goal is not to wrap 
> existing legacy systems.

There are lots of things it's good for - the same as the Unix "mount"
command - if you're into that kind of thing.  If not, it's just a plugin. 
You're free to not use it, and then everything is attached to the root.

Have fun,

Avery