[systemd-devel] systemd's connections to /run/systemd/private ?

[Zbigniew Jędrzejewski-Szmek]

On Wed, Jul 10, 2019 at 09:51:36AM -0400, Brian Reichert wrote:
> On Wed, Jul 10, 2019 at 07:37:19AM +0000, Zbigniew J??drzejewski-Szmek wrote:
> 
> > It's a bug report as any other. Writing a meaningful reply takes time
> > and effort. Lack of time is a much better explanation than ressentiments.
> 
> I wasn't expressing resentment; I apologize if it came off that way.
> 
> > Please always specify the systemd version in use. We're not all SLES
> > users, and even if we were, I assume that there might be different
> > package versions over time.
> 
> Quite reasonable:
> 
>   localhost:/var/tmp # cat /etc/os-release
>   NAME="SLES"
>   VERSION="12-SP3"
>   VERSION_ID="12.3"
>   PRETTY_NAME="SUSE Linux Enterprise Server 12 SP3"
>   ID="sles"
>   ANSI_COLOR="0;32"
>   CPE_NAME="cpe:/o:suse:sles:12:sp3"
> 
>   localhost:/var/tmp # rpm -q systemd
>   systemd-228-142.1.x86_64

That's ancient... 228 was released almost four years ago.

> > > When we first spin up a new SLES12 host with our custom services,
> > > the number of connections to /run/systemd/private numbers in the
> > > mere hundreds. 
> 
> > That sounds wrong already. Please figure out what those connections
> > are. I'm afraid that you might have to do some debugging on your
> > own, since this issue doesn't seem easily reproducible.
> 
> What tactics should I employ?  All of those file handles to
> /run/systemd/private are owned by PID 1, and 'ss' implies there are
> no peers.
> 
> 'strace' in pid shows messages are flowing, but that doesn't reveal
> the logic about how the connections get created or culled, nor who
> initiated them.

strace -p1 -e recvmsg,close,accept4,getsockname,getsockopt,sendmsg -s999

yields the relevant info. In particular, the pid, uid, and guid of the
remote is shown. My approach would be to log this to some file, and
then see which fds remain, and then look up this fd in the log.
The recvmsg calls contain the serialized dbus calls, a bit messy but
understandable. E.g. 'systemctl show systemd-udevd' gives something
like this:

recvmsg(20, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="l\1\4\1\5\0\0\0\1\0\0\0\257\0\0\0\1\1o\08\0\0\0", iov_len=24}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = 24
recvmsg(20, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="/org/freedesktop/systemd1/unit/systemd_2dudevd_2eservice\0\0\0\0\0\0\0\0\3\1s\0\6\0\0\0GetAll\0\0\2\1s\0\37\0\0\0org.freedesktop.DBus.Properties\0\6\1s\0\30\0\0\0org.freedesktop.systemd1\0\0\0\0\0\0\0\0\10\1g\0\1s\0\0\0\0\0\0\0", ...

HTH,
Zbyszek