[systemd-devel] umount fails on system with huge (2TiB) buff/cache

[Holger Kiehl]

Hello,

On Tue, 30 Jan 2024, Michal Koutný wrote:

> Hello.
> 
> On Fri, Jan 26, 2024 at 12:13:34PM +0000, Holger Kiehl <Holger.Kiehl at dwd.de> wrote:
> ...
> > Note it states 'no limit' and one can see after some minutes it says
> > it umounted /mnt/u2:
> ...
> > Confused here since it stated on serial console output
> > 
> >    [  OK  ] Unmounted /mnt/u2.
> 
> Any chance your mount unit has LazyUnmount=yes?
> 
I just checked, it is not set:

   systemctl show mnt-u2.mount|grep LazyUnmount
   LazyUnmount=no

> > The only way I can get the system to reboot properly is when sending the
> > following command before doing the reboot:
> > 
> >    echo 1 > /proc/sys/vm/drop_caches
> 
> How long does this take BTW? (Around those 10 minutes?)
> 
Yes. And I have the feeling the longer the system is running the
longer it takes. I remember seeing values of 15 minutes.

> > Is it possible to tell systemd-shutdown to wait longer or are there
> > some other parameters I need to tune?
> 
> systemd-shutdown uses sum of values from
> /proc/meminfo:{NFS_Unstable,Writeback,Dirty} to determine whether the sync
> progresses. Something in block/FS layer may got stuck if it doesn't
> apparently decrease.
> 
These values seem low if I look now:

   grep -E 'NFS_Unstable|Writeback:|Dirty' /proc/meminfo
   Dirty:            142600 kB
   Writeback:           204 kB
   NFS_Unstable:          0 kB

Just in case, here is what cat /proc/meminfo shows:

   MemTotal:       2100404416 kB
   MemFree:         6588452 kB
   MemAvailable:   2071304740 kB
   Buffers:        16754436 kB
   Cached:         1987217352 kB
   SwapCached:           12 kB
   Active:          4265420 kB
   Inactive:       2000405976 kB
   Active(anon):     752288 kB
   Inactive(anon):      280 kB
   Active(file):    3513132 kB
   Inactive(file): 2000405696 kB
   Unevictable:      176784 kB
   Mlocked:          176784 kB
   SwapTotal:      23238652 kB
   SwapFree:       23231484 kB
   Zswap:                 0 kB
   Zswapped:              0 kB
   Dirty:           1127136 kB
   Writeback:            56 kB
   AnonPages:        876600 kB
   Mapped:           196720 kB
   Shmem:             36580 kB
   KReclaimable:   70815728 kB
   Slab:           80465432 kB
   SReclaimable:   70815728 kB
   SUnreclaim:      9649704 kB
   KernelStack:       48720 kB
   PageTables:        13416 kB
   SecPageTables:         0 kB
   NFS_Unstable:          0 kB
   Bounce:                0 kB
   WritebackTmp:          0 kB
   CommitLimit:    1073440860 kB
   Committed_AS:    2024660 kB
   VmallocTotal:   34359738367 kB
   VmallocUsed:      522548 kB
   VmallocChunk:          0 kB
   Percpu:           708608 kB
   HardwareCorrupted:     0 kB
   AnonHugePages:         0 kB
   ShmemHugePages:        0 kB
   ShmemPmdMapped:        0 kB
   FileHugePages:         0 kB
   FilePmdMapped:         0 kB
   CmaTotal:              0 kB
   CmaFree:               0 kB
   Unaccepted:            0 kB
   HugePages_Total:       0
   HugePages_Free:        0
   HugePages_Rsvd:        0
   HugePages_Surp:        0
   Hugepagesize:       2048 kB
   Hugetlb:               0 kB
   DirectMap4k:      246592 kB
   DirectMap2M:    16109568 kB
   DirectMap1G:    2118123520 kB

Regards,
Holger