<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=Windows-1252"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang="en-SE" link="blue" vlink="purple" style="word-wrap:break-word"> <div class="WordSection1"> Thank you for your comprehensive answer and your time. I have a good understanding. <o:p></o:p> <o:p> </o:p> /proc/net/unix with xattr sounds like a good idea. I guess one caveat is that it won’t show the activatable varlinks. <o:p></o:p> <o:p> </o:p> Maybe we are talking about 2 kinds of discovery, one of them is the service discovery and the other one is the interface discovery (registry of which services implement what interface). Maybe the service discovery can happen on the file system level (any file in /run/varlink folder is a varlink service which should reply to Describe interface). I don’t think having a process to resolve the good name to socket makes sense for something this simple. <o:p></o:p> <o:p> </o:p> Maybe the interface discovery should happen on the interested services address space. For example, an API that watches and traverses through the service discovery folder and asks each varlink about their interfaces to create a registry. The interface discovery curious service pays the price of keeping a registry. Maybe the API can be part of the sd-varlink in libsystemd. <o:p></o:p> <o:p> </o:p> Umut<o:p></o:p> <o:p> </o:p> <o:p> </o:p> <div id="mail-editor-reference-message-container"> <div> <div> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"> From: Lennart Poettering <lennart@poettering.net> Date: Thursday, 28 November 2024 at 10:39 To: Umut Tezduyar Lindskog <Umut.Tezduyar@axis.com> Cc: systemd-devel@lists.freedesktop.org <systemd-devel@lists.freedesktop.org> Subject: Re: [systemd-devel] systemd and varlink<o:p></o:p> </div> <div> On Mi, 27.11.24 19:01, Umut Tezduyar Lindskog (Umut.Tezduyar@axis.com) wrote: > Hello systemd, > > We are closely observing the varlink development and excited about > it. Our guys who were at the all systems go conference mentioned > that the Resolver is not part of the systemd family. Ref: Resolver > (<a href="https://varlink.org/">https://varlink.org/</a>) . What is the upstream’s thoughts regarding > service discovery? So right now in systemd if you want to talk to a service, you just need to know the socket you have to talk to. Sockets are typically AF_UNIX sockets in the file system, hence you have pretty expressive identifiers: file system paths. For example you need to know that hostnamed's socket is /run/systemd/io.systemd.Hostname, and then you can for example issue a command like the following from the command line: varlinkctl introspect /run/systemd/io.systemd.Hostname or call a method: varlinkctl call /run/systemd/io.systemd.Hostname io.systemd.Hostname.Describe '{}' if we'd implement Varlink service discovery this would allow two things: 1. It would allow you to fire off varlink calls without specifying a socket at all, you could then do something like this: varlinkctl call - io.systemd.Hostname.Describe '{}' or so (this is purely hypothetical, this is not implemented after all). And then it would be able to use service discovery to find the right socket to talk to. Which is great, because it means everything can be figured out from the fully qualified method name already, which would make varlink even nicer to use I guess than D-Bus, because having a single identifier for a method instead of the complexity that D-Bus brings for this (i.e. to invoke a method on dbus you need to know which bus, which service name, which object path, which interface, which member name, i.e. a quintuplet of information instead of just a single identifier). 2. In theory, you could get a somewhat comprehensive list of all relevant sockets. So in a way it would be a bit like "cat /proc/net/unix" but only showing sockets that you can actually talk Varlink on. Kinda. More or less. Now, in systemd we currently haven#t bothered with implementing the service discovery part of Varlink for a variety of reasons: 1. We want to use Varlink during earliest boot already, that's one of the fundamental goals we had with all this. i.e. before we start any services, but the service discovery daemon of course would be a service, hence we'd be back in that world where we are in dbus where dbus must be up for us to do IPC, which is not an improvement. 2. One of the first places we used Varlink was the userdb stuff, i.e. this stuff: <a href="https://systemd.io/USER_GROUP_API/">https://systemd.io/USER_GROUP_API/</a> – But for that the concept implies you have a multitude of services all implementing the very same interfaces, so that multiple subsystems can provide user records to the system. But that conceptually doesn't really fit into the varlink service discovery model, which assumes services are singletons: each interface has exactly one service that implements it. It after all translates one interface name into exactly one socket address. 3. In the Linux world things are often split up between per-system and per-user services, and we'd thus need two resolvers, and thus you'd probably not really get away with only needing to specify a single fully qualified name to invoke a method, you'd also need to specifiy a scope after all, hence the gain is not as big as one might think. 4. The service discovery part is a bit underspecified, i.e. the address format is not part of the spec. Neither is the address the resolver listens on specified. Or to summarize: I think the service discovery is not really necessary for our usecase, and it probably needs more love and spec'ing out to be truly useful. I am not ruling out implementing this eventually. But I think for now we are fine with expecting specification of both a socket path and a fully qualified method name for doing method calls. Something I'd really love to see if we some day could teach varlinkctl to actually enumerate /proc/net/unix and then filter out sockets that aren't Varlink. We could for example use xattrs on the entrypoint inode for that: i.e. unix sockets in the fs that have the "user.varlink" xattr set to "1" would be discoverable as Varlink sockets. That would be super nice, because we'd use basic OS concepts only, and it would just work, and would be extensible to other protocols too. Alas – the Linux kernel VFS currently hard refuses setting user.* xattrs on socket inodes. It's an artificial limitation afaics, so I have hopes this is eventually lifted in the kernel, but right now, it's there. > >From the NEWS: > > “systemd-machined gained a pretty complete set of Varlink APIs > exposing its functionality. This is an alternative to the > pre-existing D-Bus interface.” > > For example, how can systemd-machined be discovered? machined's varlink socket is /run/systemd/machine/io.systemd.Machine, and this is even documented in its man page. Use "varlinkctl introspect /run/systemd/machine/io.systemd.Machine" to see what it offers. > If the varlink interface is an alternative to the D-Bus, how is the > authorization handled? Authorization in Varlink is two-fold: 1. There's file ownership + ACLs on the socket entrypoint inode. 2. And there's polkit, just like for dbus. systemd authenticates varlink client with polkit in various places. Right now this means talking D-Bus for this, but hopefully polkit one day learns varlink too, so that we can avoid this. But this should be invisble to clients, thankfully. (This implies that in systemd during early boot, until D-Bus + polkit are accessible no polkit authentication can take place, which hence means systemd-provided varlink services only allow root to access them then – which should be fine however, given that regular users should not be able to log in that early anyway.) > The JSON transcoding is great for debugging but it is consuming CPU > cycles in production, especially important for constraint > devices. Has there been any discussion on using alternative > transcoding but switching to JSON with monitor connections. Uh. That's made up, sorry. for large data, yes it can be slower than binary marshallings, but for smaller data (which varlink traffic is) this is barely measurable. The marshalling thing is often brought up by D-Bus apologists, but it's a bogus argument. The fact that JSON demarshalling is a bit slower than binary encodings is absolutely dwarfed by the slowness that D-Bus brings with it because it forces so many roundtrips and a single data pipe through the broker process. Roundtrips kill your performance, JSON marshalling is entirely irrelevant for that. People have done profiling on this. For example zbus' Zeeshan Ali posted about this on Mastodon some time ago, where he compared zbus and varlink demarshalling and indeed noticed that for small messages it's impossible to measure the marshalling difference between varlink and dbus, but the roundtrip diff very much is. Lennart -- Lennart Poettering, Berlin<o:p></o:p> </div> </div> </div> </div> </div> </body> </html>