Soliciting feature requests for development of an LVM library / API

[David Zeuthen]

Hi,

Sorry for not replying earlier, I wanted to reply, but typing up a
coherent and hopefully useful reply turned out to take some time

On Mon, 2008-11-24 at 14:52 -0500, Dave Wysochanski wrote:
> I'm a developer on the LVM2 project and am working on an LVM library /
> API.  Currently we're coming up with a list of requirements by querying
> various projects which either a) use the current LVM CLI directly and
> parse output, or b) might be interested in using a real LVM library.
> 
> Does this project have use for an LVM2 library/API, and if so, are there
> specific features you need?
> 
> If you have currently code that calls the LVM CLI, is this good enough?
> Or would there be something we could add in an LVM API that would be
> compelling enough for you to write to it?

There is not yet support for LVM2 in DeviceKit-disks though we probably
want to add support for it at some point (a lot of people ask for it).

Since LVM2 is built upon device-mapper, it's worth pointing out that we
currently support the dm-crypt target of device-mapper for LUKS
encrypted block devices, e.g.

 http://hal.freedesktop.org/docs/DeviceKit-disks/Device.html
 http://people.freedesktop.org/~david/gdu-create-size.png
 http://people.freedesktop.org/~david/gdu-encryped.png

It's also worth pointing out that we currently support Linux Software
RAID which is similar in complexity (at least from an user interface
point of view) to LVM

 http://people.freedesktop.org/~david/gdu-raid5.png

To start out, one of the key points about DeviceKit-disks is that it
intends to a) make all information about a block device easily available
in a structured uniform format; and b) make available operations to
non-root users using PolicyKit for authorization. All this is done via
D-Bus; c) use existing tools (like mdadm(1) and lvm(1)) where available.

Another goal is to make the integration really non-intrusive. As we
don't want to expose in DeviceKit-disks each and every exotic operation
supported by the existing tools (since many rarely makes sense in a UI),
we want to be able to tell users to just keep using the native tools and
just make sure DeviceKit-disks can cope properly with other programs
making changes. We'd need this anyway, people (including myself) *are*
going to want to use the command line and not the UI.

This raises the bar for subsystems/tools we want to support; change
notification needs to be a key element in that.

I've looking into supporting LVM into DeviceKit-disks on several
occasions; here's an analysis.

First, it's a problem that device-mapper (currently) has poor
integration with the rest of the Linux system; for example device-mapper
creates it's own device nodes rather than letting udev do this.This is a
problem since it causes race conditions and makes it really hard for to
hook into events via standard udev mechanisms

It's also problem that device information isn't available in standard
places like sysfs or the udev database. I realize that most of this
information is available via the vgdisplay(1), lvdisplay(1) and
pvdisplay(1) command line tools but performance and stability concerns
makes this option unattractive:

 - having a daemon, for every block device discovered, call into a
   library/program (that potentially opens all block devices) to find 
   meta data is not going to work in production. We tried similar things
   with HAL for probing for file system signatures and it's not
   going to work

So for each mapped device-mapper block device (resp. LVM), information
about each mapped device (resp. PV and LV) needs to be available either
in sysfs or the udev database. 

Also, the upstream mechanism (e.g. LVM and device-mapper) needs to ship
with the appropriate udev rules / whatever such that this information is
available and such information needs to be part of the API/ABI contract
you give users of your mechanism. 

For the record there's a bug about it here

https://bugzilla.redhat.com/show_bug.cgi?id=438604

Anything else is going to be hard to support since we aim to support
more than just Fedora / Red Hat systems where assumptions about what
patches etc. is in the device-mapper and LVM2 RPMs. We just want to tell
downstream distributors "you need device-mapper ver XYZ and LVM ver ABC
for this to work"

For the record, since we don't currently have this feature and we
currently support device-mapper for the dmcrypt target in
DeviceKit-disks we install our own program, see

http://cgit.freedesktop.org/DeviceKit/DeviceKit-disks/tree/src/devkit-disks-dm-export.c

called by our own udev rule to pick up this information. See [1] for an
example of information put in the udev database.

It's a good question what sort of information I'd like you to make
available in sysfs or the udev database. I can provide that on request
but just a note that information is needed for both PV's and LV's.
Basically most of the information you today can get from pvdisplay(1),
lvdisplay(1) and pvdisplay(1). I'm asking for this to be available in
the udev database (or better, sysfs) and properly documented.

Up until now, I've talked mostly about just information, e.g. pointing
out that it's desirable to have information about the PV's and LV's in
the standard device databases on Linux (e.g. udev and/or sysfs).

For more advanced operations, there's stuff like mirroring and other
state that one may be interested in tracking. Again, stuff like this
needs to use standard Linux mechanism like udev or sysfs. E.g. for a
mirrored block device, we need to be able to find (through udev and
sysfs) the state of each component (rebuilding, spare, idle etc.) and
the state of the mirror itself (running, ready, rebuilding, rebuild
progress / speed). We also want 'change' uevents when this information
change. In other words, we need something like 

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/md.txt;hb=HEAD

for device-mapper / LVM. Having to connect to a user space daemon
(through a library perhaps) is just asking for trouble.

So having this information would enable us to actually do sensible
things in DeviceKit-disks which would then allow e.g. GNOME applications
like Palimpsest or the File Manager to provide a meaningful and useful
experience for the end user; e.g. provide a notification of some sort if
a mirror is failing. Like we can do for Linux Software RAID.

Now, I hope this answers your question about what kind of API is desired
to be able to interact with LVM; at least it should provide some kind of
sketch and I'm sure you've got most of these planned in the library you
are talking about. 

The main point, however, is that we don't want to use a *library* to
interact with LVM; we want LVM to integrate with the standard mechanisms
on Linux (sysfs and/or udev) to provide this information. And then use
these simple existing mechanisms (sysfs fs, udev db, uevents) to make
the desired information available in DeviceKit-disks to the desktop over
D-Bus

A good question is why *don't* just use a library? Well, we tried this
route with HAL already and it turns *collecting* information (as opposed
to just providing it via a simple D-Bus API) in more than one place is
going to cause a lot of problems. First of all there's the performance
concerns outlined earlier, second there's all the race conditions, third
there's the usual complexity problems of having one big code base use a
potentially complicated library.

I hope this clarifies. And sorry for replying so late and writing such a
long reply. Thanks.

     David


[1] : Just for the record, this is namespaced with DKD_ to avoid
collisions in the future

[root at x61 ~]# udevadm info --query=all --name=/dev/dm-0|grep DKD_DM
E: DKD_DM_NAME=luks-sda2
E: DKD_DM_STATE=ACTIVE
E: DKD_DM_TABLE_STATE=LIVE
E: DKD_DM_OPENCOUNT=2
E: DKD_DM_LAST_EVENT_NR=0
E: DKD_DM_MAJOR=253
E: DKD_DM_MINOR=0
E: DKD_DM_TARGET_COUNT=1
E: DKD_DM_TARGET_TYPES=crypt

[root at x61 ~]# udevadm info --query=all --name=/dev/dm-1|grep DKD_DM
E: DKD_DM_NAME=VolGroup00-LogVol01
E: DKD_DM_UUID=LVM-ixdFcxPAV4TKy3LCDpqlAVO6xdkbpUe8scHIBAK1HnBE4nLDdOLYgFNJcdQXSPij
E: DKD_DM_STATE=ACTIVE
E: DKD_DM_TABLE_STATE=LIVE
E: DKD_DM_OPENCOUNT=1
E: DKD_DM_LAST_EVENT_NR=0
E: DKD_DM_MAJOR=253
E: DKD_DM_MINOR=1
E: DKD_DM_TARGET_COUNT=1
E: DKD_DM_TARGET_TYPES=linear

[root at x61 ~]# udevadm info --query=all --name=/dev/dm-2|grep DKD_DM
E: DKD_DM_NAME=VolGroup00-LogVol00
E: DKD_DM_UUID=LVM-ixdFcxPAV4TKy3LCDpqlAVO6xdkbpUe86lDwlBm2WidkTJjIbj8I2QF4rY0jAxYU
E: DKD_DM_STATE=ACTIVE
E: DKD_DM_TABLE_STATE=LIVE
E: DKD_DM_OPENCOUNT=0
E: DKD_DM_LAST_EVENT_NR=0
E: DKD_DM_MAJOR=253
E: DKD_DM_MINOR=2
E: DKD_DM_TARGET_COUNT=1
E: DKD_DM_TARGET_TYPES=linear