Configuration API

[Dave Cridland [Home]]

On Tue Apr 27 16:40:59 2004, Sean Middleditch wrote:
> Why don't we come up with a list of use cases and needs for the
> configuration API first?  Then come up with an API that meets all those
> needs.

Oh, goodie. That's what I was going to suggest, in different terms:

Rather than worry about a common API at this stage, we should, I feel, worry 
about a data model. Once we have that, then let's worry about a namespace 
arrangement. Then what kind of access an API needs to provide. Then an API, 
if we feel like it.

For the rest of this email, I'll largely play the part of ACAP protagonist.

For those that don't know, ACAP is RFC2244, a proposed standard IETF protocol 
for storage and access of configuration data, where 'configuration data' was 
intended from the start to cover stuff like addressbooks, bookmarks, et al, 
as well as simple preferences. It's been barely used and deployed, but a huge 
amount of thought went into it from a fair number of different organizations, 
so I think it can serve us well as prior art, at least.

(I personally think it can serve us well as an actual system, but I am, as 
usual, the only one who thinks that, and thus the only person maintaining, or 
at least trying to maintain, an actual ACAP implementation.)

FWIW, GConf has evolved such that it's surprisingly close to ACAP, 
essentially a handful of different choices and some simplification. And 
CORBA, of course. This, to me, both shows that GConf is a good system, and 
vindicates a lot of the work done with ACAP.

> 
> Some apps need to store complex data types, for example.  (think
> structs, or maybe just associative arrays.)  Do we want lock down and

ACAP has datasets (think GConf's directories), but where most systems have 
keys, ACAP has entries, which have multiple attributes.

The dataset class for general preferences has, aside from the standard 
attributes such as 'modtime', simply one attribute, normally, called 
'option.value', although provision is there for more complex structures, 
especially in the vendor namespaces.

Most dataset classes, however, have multiple attributes defined.

The examples in the option dataset class draft relating to suggestions for 
when to use structured options are things like colour definitions.

> 
> read-only preferences?  Have to make sure the API can support those;

ACAP has ACLs everywhere. Specifically, any attribute may have an ACL, 
there's an overriding ACL for both datasets and entries, and there are 
default ACLs for every attribute, and a final fallback per dataset.

> 
> apps need notification when preferences change between read only and
> read write and we probably want API for privileged users/processes to

ACAP has notifications, and they would indeed be provided if any ACL were 
changed. Or should be. I'm not sure if my implementation always does, now you 
mention it.

ACAP's notifications are rather more complex than any other system I've seen. 
You can, for instance, ask to be kept informed about all entires which fall 
in the first thirty entries to have fish in the first name (when sorted as if 
the alias were a number), and also have cat in the last name, sorting by 
email address, but only the first ten of those.

Admittedly, that's a highly contrived example, but if you've an addressbook 
viewer which is showing your addressbook in alphabetical order, it's pretty 
easy to have the ACAP server give you the list of addresses currently 
visible, and keeping that list up to date for you.

> 
> modify system defaults or lock-down state in a backend-neutral way.
> 

I don't actually think that's a different, or extended, API, actually, since, 
if you use an ACL model, anyone should be able to change them - assuming they 
have the relevant rights, of course.

As to defaults, my view is that you have:

1) Hard-coded settings, which the app or system applies when no value is 
found.
2) 'shipped defaults', which the distribution provide.
3) Site defaults, which the administrator provides as needed.
4) Group defaults, again most likely provided by the administrator.
5) See (4), if you have nested groups. And loop as much as you like.
6) User settings.

This maps closely to what ACAP provides, which is (3)->(6), all 
> pre-calculated, as well as what GConf provides which is {2,3,6}, except 
GConf seperates out mandatory settings.

Any level between 3 and 6 might be readonly to the user. Or indeed the user 
may not have any rights, in which point 1 or 2 would take over.

> 
> Is this API going to be only actual preferences, or do we also want to
> store application state (window position and such) and data (recent
> files?) as well?  If yes, the API needs to be able to handle the
> oddities of each.  (We don't want a settings-oriented LDAP backend to
> store state because that changes very often and LDAP is slow for
> writing, for example.)

I'd argue that these are seperate problems. I'd also argue LDAP sucks for 
settings data of any kind.

I'd love them to be solved in a cool, platform neutral way, though.

I'd think it was simply scrummy to logout of my Linux box, wander over to a 
Windows machine, log in, and find an email client open because I left one 
open on the Linux box. Switching from GNOME to KDE could also do much the 
same thing - it might switch the apps I was using, but would share as much 
session data as possible.

I'm pipe dreaming, though.

The same data store could be used for both, but I can't see a reason why the 
same API needs to be used.

> 
> Do we want/need temporary locks?  For example, if the user is editing
> setting /foo/bar in app A, should the API allow it to mark that setting
> as "being edited" so that app B won't allow the user to start changing
> it there (and causing potential confusing).
> 

I'd highly recommend a store-if-not-modified-since method. This means that 
all locks can be on the daemon, if anywhere, and certainly there's no 
application controlled locking, and no races. Again, this is an ACAP idea.

I say no races, although in the strict definition of a race condition - that 
the outcome depends on the precise timing of events - there *is* a race, it's 
just that everyone knows who won.

> 
> The API is a contract with the program which means the backend has to
> uphold that contract.  What requirements then do we put on the backend? 
> Maybe change notifications are optional.  But do commands that write
> multiple values have any kind of guarantee of atomicity?  This could be
> fairly important if we go with the above possible need of storing data
> or state using this API.  Even preferences may need this as certain
> combinations may not make sense or cause problems, so changes that
> naturally affect multiple keys must be stored atomically.

ACAP's STORE command, which is (essentially) the only method of writing data, 
can operate on multiple changes at once, all of which either work or fail 
together. To be fair, I've yet to find much use for it, but I think I 
probably will as I move towards supporting lists in the addressbook in my 
client.

It was obviously thought of as being useful enough to put in, anyway, and it 
was an easy bit of coding on the server, surprisingly. I honestly thought 
it'd be a real tough one, being, as I am, a bear of little brain.

As to which functionality needs to be available, I would argue that it's 
possible to provide notifications in any API - the question is, how timely 
are these notifications? In general, if we assume an LR style, daemon-based, 
backend, FAM won't provide notifications instantly.

However, add in a conditional store, and a method to force all updates to be 
checked for and delivered, and the question becomes somewhat moot - any time 
you're relying on the data being absolutely up-to-date, you request updates, 
and any time you're storing data based on the assumption that you kow the 
current state, you store it conditionally.

> 
> Will the API provide built-in rollback features for Undo operations in
> apps, or must each and every app reimplement this feature, with many not
> even bothering because its too much work?
> 

The problem with rollback operations is that while considering each 
application in isolation, it sounds like an easy thing to do.

The trouble is, that applications may - and if we've any luck will - share 
preferences. If app A changes preference P from 1 to 2, app B changes P from 
2 to 3, and app A subsequently 'rolls back', what happens to P? Should it be 
3, or 1? What if B used a conditional store? The short answer is that if I 
don't tell you what the semantics of P actually are, then you can't tell me. 
Worse, if I don't tell you what A and B *think* the semantics are, you can't 
tell me either.

> 
> Are schemas and data sanity/integrity checking mandatory, backend
> specific, or something for the app to worry about?
> 

For the App. Multiple versions of the same app may disagree on what they 
expect the schema to be, and should be coded to expect odd values. Besides 
which, it'd be possible for one of these awful UNIX chaps to come in with - 
dare I say it - a text editor.

In addition, imposing schema validation on a configuration system also 
imposes a significant amount of effort, which may have already been taken 
care of by the app anyway - in fact, 99 times out of 100 it will have been. 
We are, in general, talking about desktop apps, which don't usually allow the 
user to configure them in ways they don't, themselves, then understand...

ACAP, FWIW, only enforces syntax checking on attributes which are used by the 
ACAP server itself.

> 
> How will the API handle differences between per-user and per-machine
> settings, system-wide and session-wide settings, and so on?
> 

A breif tour of ACAP namespacing will shortly be departing. Please stand 
still while the brain is in motion.

ACAP operates out of datasets, as I've breifly allueded to up there. *points*.

The dataset has a path, as you'd expect. The top of the path is the dataset 
class name, such as 'addressbook', 'bookmarks', 'option', etc. That 
essentially is describing the intended semantics of the entires within.

Next, theres either one or two components which describe the scope, as it 
were, of the entries. 'user/dwd', for instance, means they're mine. 
'group/foo folk' means they belong to the group 'foo folk' - although that 
doesn't tell you if they appear as part of 'foo folk' members' options, or if 
it's a shared space for 'foo folk' everywhere.

'site' is sitewide settings - it's assumed that most datasets will, 
eventually, inherit somewhere from here.

'host/basil.example.net' would signify settings relevant to the host basil.

After those components comes the actual data itself.

It's not a huge stretch to include the session in the namespace too, which 
could have the interesting abailty to then 'copy' session state from a 
different session on the fly.

Finally, ACAP doesn't handle the case - at all - of settings that change both 
per-user *and* per-machine. For instance, on sybil I might prefer to use mutt 
as my mail client, whereas on basil I might decide that running Outlook in 
WINE was more fun. Manuel, in case you wonder, runs Windows natively.

While we're here, the dataset class can be a vendor-specific one, or it can 
be standard, and every attribute can be standard, vendor-specific, 
user-specific, or site specific, all with different prefices to avoid 
collisions.

> 
> Configuration storage is a very complex problem.  There's a reason
> nobody has it anywhere near right yet.  ;-)
> 

I think RFC2244 comes close. There's a few ikky bits in the document, as yet, 
but it's a joy to be able to have my email open on two machines at once, 
reconfigure it on one, and have the other client 'notice' immediately.

Dave.