[systemd-devel] Help understanding "notify" (SLES15 SP3 restarting smartd)

[Ulrich Windl]

Hi!

I have a support case for SLES15 SP3 (systemd-246.16-7.33.1.x86_64) where smartd is restarted for no obvious reason. Support says everything is fine, but I disagree.
Specifically smartd.service uses:

[Service]
Type=notify
EnvironmentFile=-/var/lib/smartmontools/smartd_opts
ExecStart=/usr/sbin/smartd -n $smartd_opts
ExecReload=/bin/kill -HUP $MAINPID

And mostly I wonder to what types of changes "notify" does react.
Specifically the smartd_opts was never changed, but stil lsmartd is restarted (by SIGTERM, not SIGHUP, BTW).

h16:~ # ll /var/lib/smartmontools/smartd_opts
-rw-r--r-- 1 root root 74 Feb  9 00:03 /var/lib/smartmontools/smartd_opts
h16:~ # stat /var/lib/smartmontools/smartd_opts
  File: /var/lib/smartmontools/smartd_opts
  Size: 74              Blocks: 8          IO Block: 4096   regular file
Device: 3ah/58d Inode: 34314       Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2022-02-09 11:17:21.464795678 +0100
Modify: 2022-02-09 00:03:12.000000000 +0100
Change: 2022-02-09 00:05:01.012019105 +0100
 Birth: 2020-11-27 09:05:09.871171180 +0100
The file just contains:
h16:~ # cat /var/lib/smartmontools/smartd_opts
# Generated by /usr/lib/smartmontools/generate_smartd_opts
smartd_opts=""
------
(Access was the time when I had looked into it)
Logs near 00:03:12:
Feb 09 00:03:12 h16 systemd[1]: Starting Generate issue file for login session...
Feb 09 00:03:12 h16 systemd[1]: Starting Update smartd options...
Feb 09 00:03:13 h16 systemd[1]: issue-generator.service: Succeeded.
Feb 09 00:03:13 h16 systemd[1]: Finished Generate issue file for login session.
Feb 09 00:03:13 h16 smartd[40401]: smartd received signal 15: Terminated
Feb 09 00:03:13 h16 systemd[1]: Stopping Self Monitoring and Reporting Technology (SMART) Daemon...
Feb 09 00:03:13 h16 smartd[40401]: Device: /dev/bus/0 [megaraid_disk_00] [SAT], state written to /var/lib/smartmontools/smartd...
Feb 09 00:03:13 h16 smartd[40401]: Device: /dev/bus/0 [megaraid_disk_01] [SAT], state written to /var/lib/smartmontools/smartd...
Feb 09 00:03:13 h16 smartd[40401]: smartd is exiting (exit status 0)
Feb 09 00:03:13 h16 systemd[1]: smartd.service: Succeeded.
Feb 09 00:03:13 h16 systemd[1]: Stopped Self Monitoring and Reporting Technology (SMART) Daemon.
Feb 09 00:03:13 h16 systemd[1]: Starting Self Monitoring and Reporting Technology (SMART) Daemon...
Feb 09 00:03:13 h16 smartd[11734]: smartd 7.0 2019-05-21 r4917 [x86_64-linux-5.3.18-150300.59.43-default] (SUSE RPM)

Logs near 00:05:01:
Feb 09 00:01:23 h16 systemd[1]: Started DATA-PROTECTOR-INET (172.20.16.3:44300).
...
Feb 09 00:05:11 h16 systemd[1]: omni at 168-172.20.16.16:5555-172.20.16.3:44300.service: Succeeded.

So out backup software was active in that interval, and one feature it has is to reset the access times of the files.
(In the past that feature also has ruined SUSE's cron.daily jonbs as they were querying the wrong timestamp, but refused to fix it)

Regards,
Ulrich