[systemd-devel] Q: A way to log peak memory footprint of deactivating units?

[Russell Haley]

For background, this came up when I was trying to work around Fedora's
infamous PackageKit memory leak, reported in various places:

https://github.com/PackageKit/PackageKit/issues/460
https://bugzilla.redhat.com/show_bug.cgi?id=1354074
https://bugzilla.redhat.com/show_bug.cgi?id=1854875
https://bugzilla.redhat.com/show_bug.cgi?id=1896964

I set ShutdownTimeout=60 in PackageKit.conf, somewhat more aggressive
than the 300s suggested in the github issue.  This worked great and
solved the persistent memory hogging on my machine.

However, I got curious about how much filesystem cache might be getting
forced out by the daily update checks. The log for a typical run looks
like this:

> Aug 12 19:59:14 censored systemd[1]: Starting packagekit.service - PackageKit Daemon...
> Aug 12 19:59:14 censored PackageKit[40462]: daemon start
> Aug 12 19:59:14 censored systemd[1]: Started packagekit.service - PackageKit Daemon.
> Aug 12 19:59:14 censored PackageKit[40462]: uid 1000 is trying to obtain org.freedesktop.packagekit.system-sources-refresh auth (only_trusted:0)
> Aug 12 19:59:14 censored PackageKit[40462]: uid 1000 obtained auth for org.freedesktop.packagekit.system-sources-refresh
> Aug 12 19:59:45 censored PackageKit[40462]: refresh-cache transaction /4084_eceebbea from uid 1000 finished with success after 30946ms
> Aug 12 19:59:48 censored PackageKit[40462]: get-updates transaction /4086_aedbdccb from uid 1000 finished with success after 2373ms
> Aug 12 19:59:48 censored PackageKit[40462]: get-updates transaction /4087_dcbdccac from uid 1000 finished with success after 200ms
> Aug 12 20:00:49 censored PackageKit[40462]: daemon quit
> Aug 12 20:00:49 censored systemd[1]: packagekit.service: Deactivated successfully.
> Aug 12 20:00:49 censored systemd[1]: packagekit.service: Consumed 14.698s CPU time.

Since systemd logs the total CPU time used when the unit deactivates, I
wonder if there's a way to make it log the peak memory footprint too,
kind of like the time command's "maxresident". The unit does have
MemoryAccounting=yes.

It turned out that it was simple to write a shell script loop to wait
until packagekit was activated and sample the memory usage before the
timeout expired. (About 228 MiB.)  However, I am still interested to
know if there might be a better/more general method.

P.S. Since I had to re-send this mail with the correct From: address, I
looked into it and apparently kernel 5.19 added a memory.peak to cgroups
v2, so I think it very recently become possible to have an elegant
implementation of this.

Thanks,
Russ