Soliciting feature requests for development of an LVM library / API

[Kay Sievers]

On Fri, Dec 5, 2008 at 16:24, David Zeuthen <david at fubar.dk> wrote:
> Sorry for not replying earlier, I wanted to reply, but typing up a
> coherent and hopefully useful reply turned out to take some time
>
> On Mon, 2008-11-24 at 14:52 -0500, Dave Wysochanski wrote:
>> I'm a developer on the LVM2 project and am working on an LVM library /
>> API.  Currently we're coming up with a list of requirements by querying
>> various projects which either a) use the current LVM CLI directly and
>> parse output, or b) might be interested in using a real LVM library.
>>
>> Does this project have use for an LVM2 library/API, and if so, are there
>> specific features you need?
>>
>> If you have currently code that calls the LVM CLI, is this good enough?
>> Or would there be something we could add in an LVM API that would be
>> compelling enough for you to write to it?
>
> There is not yet support for LVM2 in DeviceKit-disks though we probably
> want to add support for it at some point (a lot of people ask for it).
>
> Since LVM2 is built upon device-mapper, it's worth pointing out that we
> currently support the dm-crypt target of device-mapper for LUKS
> encrypted block devices, e.g.
>
>  http://hal.freedesktop.org/docs/DeviceKit-disks/Device.html
>  http://people.freedesktop.org/~david/gdu-create-size.png
>  http://people.freedesktop.org/~david/gdu-encryped.png
>
> It's also worth pointing out that we currently support Linux Software
> RAID which is similar in complexity (at least from an user interface
> point of view) to LVM
>
>  http://people.freedesktop.org/~david/gdu-raid5.png
>
> To start out, one of the key points about DeviceKit-disks is that it
> intends to a) make all information about a block device easily available
> in a structured uniform format; and b) make available operations to
> non-root users using PolicyKit for authorization. All this is done via
> D-Bus; c) use existing tools (like mdadm(1) and lvm(1)) where available.
>
> Another goal is to make the integration really non-intrusive. As we
> don't want to expose in DeviceKit-disks each and every exotic operation
> supported by the existing tools (since many rarely makes sense in a UI),
> we want to be able to tell users to just keep using the native tools and
> just make sure DeviceKit-disks can cope properly with other programs
> making changes. We'd need this anyway, people (including myself) *are*
> going to want to use the command line and not the UI.
>
> This raises the bar for subsystems/tools we want to support; change
> notification needs to be a key element in that.
>
> I've looking into supporting LVM into DeviceKit-disks on several
> occasions; here's an analysis.
>
> First, it's a problem that device-mapper (currently) has poor
> integration with the rest of the Linux system; for example device-mapper
> creates it's own device nodes rather than letting udev do this.This is a
> problem since it causes race conditions and makes it really hard for to
> hook into events via standard udev mechanisms
>
> It's also problem that device information isn't available in standard
> places like sysfs or the udev database. I realize that most of this
> information is available via the vgdisplay(1), lvdisplay(1) and
> pvdisplay(1) command line tools but performance and stability concerns
> makes this option unattractive:
>
>  - having a daemon, for every block device discovered, call into a
>   library/program (that potentially opens all block devices) to find
>   meta data is not going to work in production. We tried similar things
>   with HAL for probing for file system signatures and it's not
>   going to work
>
> So for each mapped device-mapper block device (resp. LVM), information
> about each mapped device (resp. PV and LV) needs to be available either
> in sysfs or the udev database.
>
> Also, the upstream mechanism (e.g. LVM and device-mapper) needs to ship
> with the appropriate udev rules / whatever such that this information is
> available and such information needs to be part of the API/ABI contract
> you give users of your mechanism.
>
> For the record there's a bug about it here
>
> https://bugzilla.redhat.com/show_bug.cgi?id=438604
>
> Anything else is going to be hard to support since we aim to support
> more than just Fedora / Red Hat systems where assumptions about what
> patches etc. is in the device-mapper and LVM2 RPMs. We just want to tell
> downstream distributors "you need device-mapper ver XYZ and LVM ver ABC
> for this to work"
>
> For the record, since we don't currently have this feature and we
> currently support device-mapper for the dmcrypt target in
> DeviceKit-disks we install our own program, see
>
> http://cgit.freedesktop.org/DeviceKit/DeviceKit-disks/tree/src/devkit-disks-dm-export.c
>
> called by our own udev rule to pick up this information. See [1] for an
> example of information put in the udev database.
>
> It's a good question what sort of information I'd like you to make
> available in sysfs or the udev database. I can provide that on request
> but just a note that information is needed for both PV's and LV's.
> Basically most of the information you today can get from pvdisplay(1),
> lvdisplay(1) and pvdisplay(1). I'm asking for this to be available in
> the udev database (or better, sysfs) and properly documented.
>
> Up until now, I've talked mostly about just information, e.g. pointing
> out that it's desirable to have information about the PV's and LV's in
> the standard device databases on Linux (e.g. udev and/or sysfs).
>
> For more advanced operations, there's stuff like mirroring and other
> state that one may be interested in tracking. Again, stuff like this
> needs to use standard Linux mechanism like udev or sysfs. E.g. for a
> mirrored block device, we need to be able to find (through udev and
> sysfs) the state of each component (rebuilding, spare, idle etc.) and
> the state of the mirror itself (running, ready, rebuilding, rebuild
> progress / speed). We also want 'change' uevents when this information
> change. In other words, we need something like
>
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/md.txt;hb=HEAD
>
> for device-mapper / LVM. Having to connect to a user space daemon
> (through a library perhaps) is just asking for trouble.
>
> So having this information would enable us to actually do sensible
> things in DeviceKit-disks which would then allow e.g. GNOME applications
> like Palimpsest or the File Manager to provide a meaningful and useful
> experience for the end user; e.g. provide a notification of some sort if
> a mirror is failing. Like we can do for Linux Software RAID.
>
> Now, I hope this answers your question about what kind of API is desired
> to be able to interact with LVM; at least it should provide some kind of
> sketch and I'm sure you've got most of these planned in the library you
> are talking about.
>
> The main point, however, is that we don't want to use a *library* to
> interact with LVM; we want LVM to integrate with the standard mechanisms
> on Linux (sysfs and/or udev) to provide this information. And then use
> these simple existing mechanisms (sysfs fs, udev db, uevents) to make
> the desired information available in DeviceKit-disks to the desktop over
> D-Bus
>
> A good question is why *don't* just use a library? Well, we tried this
> route with HAL already and it turns *collecting* information (as opposed
> to just providing it via a simple D-Bus API) in more than one place is
> going to cause a lot of problems. First of all there's the performance
> concerns outlined earlier, second there's all the race conditions, third
> there's the usual complexity problems of having one big code base use a
> potentially complicated library.
>
> I hope this clarifies. And sorry for replying so late and writing such a
> long reply. Thanks.

I share the same view.

We need proper integration of LVM with udev and the kernel, which means:
  - no device nodes/links created in /dev, if udev is active
  - at least basic device information in sysfs
  - proper userspace events from the kernel if something changes
  - dm/LVM tools export information about dm/LVM devices
     in <KEY>=<value>\n format, not other format will work

The exported keys will be imported into the udev database and also
used to create meaningful symlinks to the kernel device name. If udev
is active, the dm/LVM lib/tools must not touch /dev besides for
possible temporary nodes they delete immediately after use.
They must integrate properly with udev, and udev needs the symlink
information, /dev/mapper/ and such must be maintained by udev rules,
to be able to use them from other userspace tools which intergrate
with udev/DeviceKit.

We do not need any "smart" library interfaces, which eventually apply
built-in policies, open/investigate other devices than asked for,
parse files from other places. We need interfaces which are foremost
simple to call and debug from the commandline.

> [1] : Just for the record, this is namespaced with DKD_ to avoid
> collisions in the future
>
> [root at x61 ~]# udevadm info --query=all --name=/dev/dm-0|grep DKD_DM
> E: DKD_DM_NAME=luks-sda2
> E: DKD_DM_STATE=ACTIVE
> E: DKD_DM_TABLE_STATE=LIVE
> E: DKD_DM_OPENCOUNT=2
> E: DKD_DM_LAST_EVENT_NR=0
> E: DKD_DM_MAJOR=253
> E: DKD_DM_MINOR=0
> E: DKD_DM_TARGET_COUNT=1
> E: DKD_DM_TARGET_TYPES=crypt

We can *almost* do this now with:
  dmsetup info -c --nameprefixes --unquoted --rows --noheadings \
       -o name,uuid,suspended,readonly,major,minor,open,tables_loaded

But we still miss information like the TARGET_TYPES information. I
wonder how long it will take, the whole story takes like two years
now. :)

Thanks,
Kay