[systemd-devel] Antw: Re: Antw: failing unmounts during reboot

Mon Aug 5 06:04:49 UTC 2019

>>> Andrei Borzenkov <arvidjaar at gmail.com> schrieb am 01.08.2019 um 20:38 in
Nachricht <f8446dea-21f2-dc7a-8411-3ea6e4729c54 at gmail.com>:
> 29.07.2019 9:38, Ulrich Windl пишет:
>>>>> Ulrich Windl schrieb am 29.07.2019 um 08:23 in Nachricht <5D3E90D1.4EC :
161 
> :
>> 60728>:
>>>>>> Frank Steiner <fsteiner-mail1 at bio.ifi.lmu.de> schrieb am 25.07.2019 um
14:14 
> in
>>> Nachricht <913a3c04-a666-b44b-c6ec-fe3d8a7fe95e at bio.ifi.lmu.de>:
>>>> Ulrich Windl wrote:
>>>>
>>>>> *1: I have a support call open with SUSE:
>>>>> Before systemd (almost) all processes were killed before unmounting.
>>>>> With systemd I'm seeing excessive reboot delays due to unmount timing
out. 
>>>> For example if you have a process started from NFS that has a log file on

>>> NFS 
>>>> open, too.
>>>>> It seems the order is roughly like this:
>>>>> 1) Shutdown the network
>>>>> 2) Try unmounting filesystems, including NFS
>>>>> 3) Kill remaining processes
>>>>
>>>> I cannot confirm that, at least not for SLES/D 15. All mount units
>>>> for NFS filesystems created from fstab get "Before=remote-fs.target",
>>>> so they are shutdown before the network goes down. Check in
>>>> /run/systemd/generator to see if this entry is missing in your units.
>>>
>>> In SLES12 SP4 (originally reported for SP3) I have:
>>> # Automatically generated by systemd-fstab-generator
>>>
>>> [Unit]
>>> SourcePath=/etc/fstab
>>> Documentation=man:fstab(5) man:systemd-fstab-generator(8)
>>> Before=remote-fs.target
>>>
>>> [Mount]
>>> What=server:/exports/home
>>> Where=/home
>>> Type=nfs
>> 
>> Sorry I hit "send" too quickly:
>> That would mean the problem of not being unable to umnount /home is not
that 
> the network is down, but that some process still has open  files on /home.
>> 
>> However from the original problem report:
>> 
>> [  OK  ] Stopped target Host and Network Name Lookups. 
>> 		 Stopping Name Service Cache Daemon... 
>> [  OK  ] Stopped target Network. 
>> 		 Stopping wicked managed network interfaces... 
>> [  OK  ] Stopped Name Service Cache Daemon. 
>> [  OK  ] Stopped wicked managed network interfaces. 
>> 		 Stopping wicked network nanny service... 
>> [  OK  ] Stopped Check if the profile matches the system. 
>> [  OK  ] Stopped wicked network nanny service. 
>> 		 Stopping wicked network management service daemon... 
>> [  OK  ] Stopped wicked network management service daemon. 
>> 		 Stopping wicked DHCPv4 supplicant service... 
>> 		 Stopping wicked AutoIPv4 supplicant service... 
>> 		 Stopping wicked DHCPv6 supplicant service... 
>> [  OK  ] Stopped wicked DHCPv4 supplicant service. 
>> [  OK  ] Stopped wicked DHCPv6 supplicant service. 
>> [  OK  ] Stopped wicked AutoIPv4 supplicant service. 
>> 		 Stopping D-Bus System Message Bus... 
>> [  OK  ] Stopped SuSEfirewall2 phase 1. 
>> [  OK  ] Stopped D-Bus System Message Bus. 
>> [  OK  ] Stopped target Basic System. 
>> [  OK  ] Stopped target Sockets. 
> 
> Stopping (or at least attempt to stop) /home should have happened before
> these lines.
> 
>> ... I would call that a "network shutdown"...
>> [  OK  ] Stopped target Host and Network Name Lookups. 
>> 		 Stopping Name Service Cache Daemon... 
>> [  OK  ] Stopped target Network. 
>> 		 Stopping wicked managed network interfaces... 
>> [  OK  ] Stopped Name Service Cache Daemon. 
>> [  OK  ] Stopped wicked managed network interfaces. 
>> 		 Stopping wicked network nanny service... 
>> [  OK  ] Stopped Check if the profile matches the system. 
>> [  OK  ] Stopped wicked network nanny service. 
>> 		 Stopping wicked network management service daemon... 
>> [  OK  ] Stopped wicked network management service daemon. 
>> 		 Stopping wicked DHCPv4 supplicant service... 
>> 		 Stopping wicked AutoIPv4 supplicant service... 
>> 		 Stopping wicked DHCPv6 supplicant service... 
>> [  OK  ] Stopped wicked DHCPv4 supplicant service. 
>> [  OK  ] Stopped wicked DHCPv6 supplicant service. 
>> [  OK  ] Stopped wicked AutoIPv4 supplicant service. 
>> 		 Stopping D-Bus System Message Bus... 
>> [  OK  ] Stopped SuSEfirewall2 phase 1. 
>> [  OK  ] Stopped D-Bus System Message Bus. 
>> [  OK  ] Stopped target Basic System. 
>> [  OK  ] Stopped target Sockets. 
> 
> Do you really have identical lines second time in your log?

No, I guess it was a double-paste error on my side.

> 
> You need to provide full "systemctl show home.mount" and complete log
> from boot to shutdown.

Well, it just looked too complex to me (complete logs maybe next time I boot:
Where=/home
What=server:/exports/home
Options=rw,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,proto=t
Type=nfs4
TimeoutUSec=1min 30s
ControlPID=0
DirectoryMode=0755
SloppyOptions=no
Result=success
Slice=system.slice
ControlGroup=/system.slice/home.mount
MemoryCurrent=18446744073709551615
CPUUsageNSec=18446744073709551615
TasksCurrent=0
Delegate=no
CPUAccounting=no
CPUShares=18446744073709551615
StartupCPUShares=18446744073709551615
CPUQuotaPerSecUSec=infinity
BlockIOAccounting=no
BlockIOWeight=18446744073709551615
StartupBlockIOWeight=18446744073709551615
MemoryAccounting=no
MemoryLimit=18446744073709551615
DevicePolicy=auto
TasksAccounting=yes
TasksMax=512
UMask=0022
LimitCPU=18446744073709551615
LimitFSIZE=18446744073709551615
LimitDATA=18446744073709551615
LimitSTACK=18446744073709551615
LimitCORE=18446744073709551615
LimitRSS=18446744073709551615
LimitNOFILE=4096
LimitAS=18446744073709551615
LimitNPROC=3490
LimitMEMLOCK=65536
LimitLOCKS=18446744073709551615
LimitSIGPENDING=3490
LimitMSGQUEUE=819200
LimitNICE=0
LimitRTPRIO=0
LimitRTTIME=18446744073709551615
OOMScoreAdjust=0
Nice=0
IOScheduling=0
CPUSchedulingPolicy=0
CPUSchedulingPriority=0
TimerSlackNSec=50000
CPUSchedulingResetOnFork=no
NonBlocking=no
StandardInput=null
StandardOutput=journal
StandardError=inherit
TTYReset=no
TTYVHangup=no
TTYVTDisallocate=no
SyslogPriority=30
SyslogLevelPrefix=yes
SyslogLevel=6
SyslogFacility=3
SecureBits=0
CapabilityBoundingSet=18446744073709551615
MountFlags=0
PrivateTmp=no
PrivateNetwork=no
PrivateDevices=no
ProtectHome=no
ProtectSystem=no
SameProcessGroup=yes
UtmpMode=init
IgnoreSIGPIPE=yes
NoNewPrivileges=no
SystemCallErrorNumber=0
RuntimeDirectoryMode=0755
KillMode=control-group
KillSignal=15
SendSIGKILL=yes
SendSIGHUP=no
Id=home.mount
Names=home.mount
Requires=-.mount system.slice
Wants=network-online.target
RequiredBy=remote-fs.target
Conflicts=umount.target
Before=remote-fs.target umount.target
After=nfs-server.service systemd-journald.socket system.slice network.target
rem
RequiresMountsFor=/
Documentation=man:fstab(5) man:systemd-fstab-generator(8)
Description=/home
LoadState=loaded
ActiveState=active
SubState=mounted
FragmentPath=/run/systemd/generator/home.mount
SourcePath=/etc/fstab
UnitFileState=bad
UnitFilePreset=disabled
InactiveExitTimestamp=Tue 2019-07-16 15:45:43 CEST
InactiveExitTimestampMonotonic=26902098
ActiveEnterTimestamp=Tue 2019-07-16 15:45:43 CEST
ActiveEnterTimestampMonotonic=26902098
ActiveExitTimestampMonotonic=0
InactiveEnterTimestampMonotonic=0
CanStart=yes
CanStop=yes
CanReload=yes
CanIsolate=no
StopWhenUnneeded=no
RefuseManualStart=no
RefuseManualStop=no
AllowIsolate=no
DefaultDependencies=yes
OnFailureJobMode=replace
IgnoreOnIsolate=yes
NeedDaemonReload=no
JobTimeoutUSec=0
JobRunningTimeoutUSec=0
JobTimeoutAction=none
ConditionResult=no
AssertResult=no
ConditionTimestampMonotonic=0
AssertTimestampMonotonic=0
Transient=no
NetClass=0

> 
>> ...
>> I don't see unmounting of /home at all. The unmount errors reported were:
>> ...
>> [  OK  ] Unmounted Lock Directory. 
>> [FAILED] Failed unmounting Runtime Directory. 
>> 		 Unmounting /var... 
>> [  OK  ] Unmounted /opt. 
>> [  OK  ] Stopped File System Check on /dev/v04/opt. 
>> [FAILED] Failed unmounting /var. 
>> [  OK  ] Stopped File System Check on /dev/v04/var. 
>> (The /var thing has different reasons)
>> ...
>> [  OK  ] Stopped Remount Root and Kernel File Systems. 
>> [  OK  ] Reached target Shutdown. 
>> ...At this point nothing more happened...
>> Shutdown did not complete within two and a half minute.
>> 
> 
> And what evidence you have that it is related to /home not being unmounted?

It seemed to work smootely until I had a background process running that had
open files on /home

> 
> _______________________________________________
> systemd-devel mailing list
> systemd-devel at lists.freedesktop.org 
> https://lists.freedesktop.org/mailman/listinfo/systemd-devel 