[systemd-devel] fstab generator/capture.mount randomly fails which depend of cryptsetup.target
Sumit Kumar
sumit.kumar1 at trellix.com
Thu Jan 16 08:48:07 UTC 2025
Hi ,
Our Linux based system uses an external storage disk called capture.
The capture disk is encrypted with LUKS to protect stored data. During
system boot, system uses the LUKS utilities to decrypt the disk before
mounting the file systems containing the stored data. Due to an issue with
inconsistent ordering of decryption and mounting services during system
bootup, during reboots the system with capture disk can get into emergency
mode.
Do we have any configuration problem ?
*System details, logs and unit files details : *
systemd version: 252
Used distribution : AlmaLinux 9.4
Linux kernel version used : 5.15.160-1.11.10.0.x86_64
Journalctl -xb output for error message:
Jan 10 09:53:04 ps systemd[1]: Listening on udev Control Socket.
Jan 10 09:53:04 ps systemd[1]: Listening on udev Kernel Socket.
Jan 10 09:53:04 ps systemd[1]: capture.mount: Directory /capture to
mount over is not empty, mounting anyway.
Jan 10 09:53:04 ps systemd[1]: Mounting /capture...
Jan 10 09:53:04 ps systemd[1]: Mounting Huge Pages File System...
Jan 10 09:53:04 ps systemd[1]: Mounting POSIX Message Queue File System...
Jan 10 09:53:04 ps systemd[1]: Mounting /sys/kernel/debug...
Jan 10 09:53:04 ps systemd[1]: Mounting Kernel Trace File System...
Jan 10 09:53:04 ps systemd[1]: Kernel Module supporting RPCSEC_GSS was
skipped because of an unmet condition check
(ConditionPathExists=/etc/krb5.keytab).
Jan 10 09:53:04 ps systemd[1]: Starting Availability of block devices...
Jan 10 09:53:04 ps systemd[1]: Starting Create List of Static Device Nodes...
Jan 10 09:53:04 ps systemd[1]: Starting Load legacy module configuration...
Jan 10 09:53:04 ps systemd[1]: Starting Monitoring of LVM2 mirrors,
snapshots etc. using dmeventd or progress polling...
Jan 10 09:53:04 ps systemd[1]: Starting Load Kernel Module configfs...
Jan 10 09:53:04 ps systemd[1]: Starting Load Kernel Module drm...
Jan 10 09:53:04 ps systemd[1]: Starting Load Kernel Module efi_pstore...
Jan 10 09:53:04 ps systemd[1]: Starting Load Kernel Module fuse...
Jan 10 09:53:04 ps systemd[1]: Starting Read and set NIS domainname
from /etc/sysconfig/network...
Jan 10 09:53:04 ps systemd[1]: First Boot Wizard was skipped because
of an unmet condition check (ConditionFirstBoot=yes).
Jan 10 09:53:04 ps systemd[1]: Rebuild Hardware Database was skipped
because of an unmet condition check (ConditionNeedsUpdate=/etc).
Jan 10 09:53:04 ps systemd[1]: systemd-journald.service: unit
configures an IP firewall, but the local system does not support
BPF/cgroup firewalling.
Jan 10 09:53:04 ps systemd[1]: (This warning is only shown for the
first unit using IP firewalling.)
Jan 10 09:53:04 ps systemd[1]: Starting Journal Service...
Jan 10 09:53:04 ps systemd[1]: Starting Load Kernel Modules...
Jan 10 09:53:04 ps systemd[1]: Starting Generate network units from
Kernel command line...
Jan 10 09:53:04 ps systemd[1]: TPM2 PCR Machine ID Measurement was
skipped because of an unmet condition check
(ConditionPathExists=/sys/firmware/efi/efivars/StubPcrKernelImage-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f).
Jan 10 09:53:04 ps systemd[1]: Create System Users was skipped because
no trigger condition checks were met.
Jan 10 09:53:04 ps systemd[1]: Starting Coldplug All udev
Devices...*Jan 10 09:53:04 ps systemd[1]: capture.mount: Mount process
exited, code=exited, status=32/n/a
Jan 10 09:53:04 ps systemd[1]: capture.mount: Failed with result
'exit-code'*.*Jan 10 09:53:04 ps systemd[1]: Failed to mount
/capture.*
Jan 10 09:53:04 ps systemd[1]: Dependency failed for Local File Systems.
Jan 10 09:53:04 ps systemd[1]: Dependency failed for Systemd service ordering.
Jan 10 09:53:04 ps systemd[1]: service-ordering.service: Job
service-ordering.service/start failed with result 'dependency'.
Jan 10 09:53:04 ps systemd[1]: Dependency failed for Create Volatile
Files and Directories.
Jan 10 09:53:04 ps systemd[1]: systemd-tmpfiles-setup.service: Job
systemd-tmpfiles-setup.service/start failed with result 'dependency'.
Jan 10 09:53:04 ps systemd[1]: Dependency failed for Mark the need to
relabel after reboot.
Jan 10 09:53:04 ps systemd[1]: selinux-autorelabel-mark.service: Job
selinux-autorelabel-mark.service/start failed with result
'dependency'.
Jan 10 09:53:04 ps systemd[1]: local-fs.target: Job
local-fs.target/start failed with result 'dependency'.
Jan 10 09:53:04 ps systemd[1]: local-fs.target: Triggering OnFailure=
dependencies.
Jan 10 09:53:04 ps systemd[1]: Unnecessary job was removed for /dev/ttyS0.
Jan 10 09:53:04 ps systemd[1]: Mounted Huge Pages File System.
*systemctl list-units --failed*
UNIT LOAD ACTIVE SUB DESCRIPTION* capture.mount loaded
failed failed /capture
LOAD = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB = The low-level unit activation state, values depend on unit type.
1 loaded units listed.
*systemctl status capture.mount*
x capture.mount - /capture
Loaded: loaded (/etc/fstab; generated)
Active: failed (Result: exit-code) since Fri 2025-01-10 09:53:03
UTC; 39min ago
Where: /capture
What: /dev/mapper/encr-sdd
Docs: man:fstab(5)
man:systemd-fstab-generator(8)
CPU: 5ms
Jan 10 09:53:04 ps mount[784]: mount: /capture: special device
/dev/mapper/encr-sdd does not exist.
Notice: journal has been rotated since unit was started, output may be
incomplete.
*systemctl cat capture.mount*# /run/systemd/generator/capture.mount#
Automatically generated by systemd-fstab-generator
[Unit]
Documentation=man:fstab(5) man:systemd-fstab-generator(8)
SourcePath=/etc/fstab
After=cryptsetup.target
Requires=cryptsetup.target
Before=local-fs.target
Requires=systemd-fsck at dev-mapper-encr\x2dsdd.service
After=systemd-fsck at dev-mapper-encr\x2dsdd.service
After=blockdev at dev-mapper-encr\x2dsdd.target
[Mount]
What=/dev/mapper/encr-sdd
Where=/capture
Type=ext4
Options=defaults,nosuid,x-systemd.requires=cryptsetup.target
*cat /etc/crypttab*
encr-sdd UUID=6af9171e-82c7-4be2-bd70-14979067727f /.encr/sdd
cat /etc/fstab
/dev/disk/by-label/boot /boot vfat ro,defaults 1 2
/dev/disk/by-label/opt /opt ext4 ro,defaults 1 2
/dev/disk/by-label/root / ext4 ro,defaults 1 1
/dev/disk/by-label/notused /notused ext4
defaults,ro,noauto 0 0
/dev/disk/by-label/rescue /rescue ext4
ro,defaults,noauto 0 0
/dev/disk/by-label/var /var ext4 defaults 1 2
/dev/disk/by-label/var_log /var/log ext4 defaults 1 2
/dev/disk/by-label/config /config ext4
defaults,nosuid 1 2
/dev/disk/by-label/swap swap swap
defaults */dev/mapper/encr-sdd /capture
ext4 defaults,nosuid,x-systemd.requires=cryptsetup.target 1 2*
/dev/mapper/v0-deferred /deferred ext4 defaults,noatime,noexec,nosuid 1 3
/dev/mapper/v0-scandir /scandir ext4
defaults,noatime,noexec,nosuid,nobarrier 1 3
/dev/mapper/v0-logs /logs ext4 defaults,noatime,noexec,nosuid 1 3
/dev/mapper/v0-wk /wk ext4 defaults,nosuid 1 3
/var/etc /etc none bind,x-systemd.requires-mounts-for=/var 0 0 0
/wk/tmp /tmp none bind,x-systemd.requires-mounts-for=/wk 0 0 0
/wk/root /root none bind,x-systemd.requires-mounts-for=/wk 0 0 0
/wk/home /home none bind,x-systemd.requires-mounts-for=/wk 0 0 0
/wk/AgentDB/db /var/McAfee/agent/db none
bind,x-systemd.requires-mounts-for=/wk 1 0 0
/dev/cdrom /media/cdrom iso9660 noexec,nosuid,noauto,owner,ro 0 0
/dev/cdrom-0 /media/cdrom-0 iso9660 noexec,nosuid,noauto,owner,ro 0 0
/dev/cdrom-usb /media/cdrom-usb iso9660 noexec,nosuid,noauto,owner,ro 0 0
none /proc/fs/nfsd nfsd noauto 0 0
nodev /sys/kernel/debug debugfs defaults 0 0
*systemctl list-unit-files | grep systemd-cryptsetup*
systemd-cryptsetup at encr\x2dsdd.service generated -
*systemctl status systemd-cryptsetup at encr\x2dsdd.service **
systemd-cryptsetup at encr\x2dsdd.service - Cryptography Setup for
encr-sdd
Loaded: loaded (/etc/crypttab; generated)
Active: inactive (dead)
Docs: man:crypttab(5)
man:systemd-cryptsetup-generator(8)
man:systemd-cryptsetup at .service(8)
*cryptsetup status encr-sdd*
/dev/mapper/encr-sdd is inactive.
*ls -lrt /dev/mapper/**
crw------- 1 root root 10, 236 Jan 10 09:53 /dev/mapper/control
lrwxrwxrwx 1 root root 7 Jan 10 09:53 /dev/mapper/v0-deferred -> ../dm-0
lrwxrwxrwx 1 root root 7 Jan 10 09:53 /dev/mapper/v0-scandir -> ../dm-1
lrwxrwxrwx 1 root root 7 Jan 10 09:53 /dev/mapper/v0-wk -> ../dm-3
lrwxrwxrwx 1 root root 7 Jan 10 09:53 /dev/mapper/v0-logs -> ../dm-2
systemctl show systemd-cryptsetup at encr\x2dsdd.service
Type=oneshot
ExitType=main
Restart=no
NotifyAccess=none
RestartUSec=100ms
TimeoutStartUSec=infinity
TimeoutStopUSec=infinity
TimeoutAbortUSec=infinity
TimeoutStartFailureMode=terminate
TimeoutStopFailureMode=terminate
RuntimeMaxUSec=infinity
RuntimeRandomizedExtraUSec=0
WatchdogUSec=infinity
WatchdogTimestampMonotonic=0
RootDirectoryStartOnly=no
RemainAfterExit=yes
GuessMainPID=yes
MainPID=0
ControlPID=0
FileDescriptorStoreMax=0
NFileDescriptorStore=0
StatusErrno=0
Result=success
ReloadResult=success
CleanResult=success
UID=[not set]
GID=[not set]
NRestarts=0
OOMPolicy=stop
ReloadSignal=1
ExecMainStartTimestampMonotonic=0
ExecMainExitTimestampMonotonic=0
ExecMainPID=0
ExecMainCode=0
ExecMainStatus=0
ExecStart={ path=/usr/lib/systemd/systemd-cryptsetup ;
argv[]=/usr/lib/systemd/systemd-cryptsetup attach encr-sdd
/dev/disk/by-uuid/6af9171e-82c7-4be2-bd70-14979067727f /.encr/sdd ;
ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ;
code=(null) ; status=0/0 }
ExecStartEx={ path=/usr/lib/systemd/systemd-cryptsetup ;
argv[]=/usr/lib/systemd/systemd-cryptsetup attach encr-sdd
/dev/disk/by-uuid/6af9171e-82c7-4be2-bd70-14979067727f /.encr/sdd ;
flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ;
status=0/0 }
ExecStop={ path=/usr/lib/systemd/systemd-cryptsetup ;
argv[]=/usr/lib/systemd/systemd-cryptsetup detach encr-sdd ;
ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ;
code=(null) ; status=0/0 }
ExecStopEx={ path=/usr/lib/systemd/systemd-cryptsetup ;
argv[]=/usr/lib/systemd/systemd-cryptsetup detach encr-sdd ; flags= ;
start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0
}
Slice=system-systemd\x2dcryptsetup.slice
ControlGroupId=0
MemoryCurrent=[not set]
MemoryAvailable=infinity
CPUUsageNSec=[not set]
TasksCurrent=[not set]
IPIngressBytes=[no data]
IPIngressPackets=[no data]
IPEgressBytes=[no data]
IPEgressPackets=[no data]
IOReadBytes=18446744073709551615
IOReadOperations=18446744073709551615
IOWriteBytes=18446744073709551615
IOWriteOperations=18446744073709551615
Delegate=no
CPUAccounting=yes
CPUWeight=[not set]
StartupCPUWeight=[not set]
CPUShares=[not set]
StartupCPUShares=[not set]
CPUQuotaPerSecUSec=infinity
CPUQuotaPeriodUSec=infinity
IOAccounting=no
IOWeight=[not set]
StartupIOWeight=[not set]
BlockIOAccounting=no
BlockIOWeight=[not set]
StartupBlockIOWeight=[not set]
MemoryAccounting=yes
DefaultMemoryLow=0
DefaultMemoryMin=0
MemoryMin=0
MemoryLow=0
MemoryHigh=infinity
MemoryMax=infinity
MemorySwapMax=infinity
MemoryLimit=infinity
DevicePolicy=auto
TasksAccounting=yes
TasksMax=50833
IPAccounting=no
ManagedOOMSwap=auto
ManagedOOMMemoryPressure=auto
ManagedOOMMemoryPressureLimit=0
ManagedOOMPreference=none
UMask=0022
LimitCPU=infinity
LimitCPUSoft=infinity
LimitFSIZE=infinity
LimitFSIZESoft=infinity
LimitDATA=infinity
LimitDATASoft=infinity
LimitSTACK=infinity
LimitSTACKSoft=8388608
LimitCORE=infinity
LimitCORESoft=0
LimitRSS=infinity
LimitRSSSoft=infinity
LimitNOFILE=524288
LimitNOFILESoft=1024
LimitAS=infinity
LimitASSoft=infinity
LimitNPROC=31771
LimitNPROCSoft=31771
LimitMEMLOCK=8388608
LimitMEMLOCKSoft=8388608
LimitLOCKS=infinity
LimitLOCKSSoft=infinity
LimitSIGPENDING=31771
LimitSIGPENDINGSoft=31771
LimitMSGQUEUE=819200
LimitMSGQUEUESoft=819200
LimitNICE=0
LimitNICESoft=0
LimitRTPRIO=0
LimitRTPRIOSoft=0
LimitRTTIME=infinity
LimitRTTIMESoft=infinity
OOMScoreAdjust=500
CoredumpFilter=0x23
Nice=0
IOSchedulingClass=2
IOSchedulingPriority=4
CPUSchedulingPolicy=0
CPUSchedulingPriority=0
CPUAffinityFromNUMA=no
NUMAPolicy=n/a
TimerSlackNSec=50000
CPUSchedulingResetOnFork=no
NonBlocking=no
StandardInput=null
StandardOutput=journal
StandardError=inherit
TTYReset=no
TTYVHangup=no
TTYVTDisallocate=no
SyslogPriority=30
SyslogLevelPrefix=yes
SyslogLevel=6
SyslogFacility=3
LogLevelMax=-1
LogRateLimitIntervalUSec=0
LogRateLimitBurst=0
SecureBits=0
CapabilityBoundingSet=cap_chown cap_dac_override cap_dac_read_search
cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap
cap_linux_immutable cap_net_bind_service cap_net_broadcast
cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module
cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct
cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time
cap_sys_tty_config cap_mknod cap_lease cap_audit_write
cap_audit_control cap_setfcap cap_mac_override cap_mac_admin
cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read cap_perfmon
cap_bpf cap_checkpoint_restore
DynamicUser=no
RemoveIPC=no
PrivateTmp=no
PrivateDevices=no
ProtectClock=no
ProtectKernelTunables=no
ProtectKernelModules=no
ProtectKernelLogs=no
ProtectControlGroups=no
PrivateNetwork=no
PrivateUsers=no
PrivateMounts=no
PrivateIPC=no
ProtectHome=no
ProtectSystem=no
SameProcessGroup=no
UtmpMode=init
IgnoreSIGPIPE=yes
NoNewPrivileges=no
SystemCallErrorNumber=2147483646
LockPersonality=no
RuntimeDirectoryPreserve=no
RuntimeDirectoryMode=0755
StateDirectoryMode=0755
CacheDirectoryMode=0755
LogsDirectoryMode=0755
ConfigurationDirectoryMode=0755
TimeoutCleanUSec=infinity
MemoryDenyWriteExecute=no
RestrictRealtime=no
RestrictSUIDSGID=no
RestrictNamespaces=no
MountAPIVFS=no
KeyringMode=shared
ProtectProc=default
ProcSubset=all
ProtectHostname=no
KillMode=control-group
KillSignal=15
RestartKillSignal=15
FinalKillSignal=9
SendSIGKILL=yes
SendSIGHUP=no
WatchdogSignal=6
*systemctl show cryptsetup.target*
Id=cryptsetup.target
Names=cryptsetup.target
Requires="systemd-cryptsetup at encr\\x2dsdd.service"
RequiredBy=capture.mount
WantedBy=sysinit.target
Before=capture.mount
After=cryptsetup-pre.target systemd-pcrphase-initrd.service
"systemd-cryptsetup at encr\\x2dsdd.service"
systemd-ask-password-console.path systemd-ask-password-wall.path
Description=cryptsetup.target
LoadState=masked
ActiveState=inactive
FreezerState=running
SubState=dead
FragmentPath=/etc/systemd/system/cryptsetup.target
UnitFileState=masked
UnitFilePreset=enabled
StateChangeTimestamp=Fri 2025-01-10 09:53:06 UTC
StateChangeTimestampMonotonic=11420678
InactiveExitTimestampMonotonic=0
ActiveEnterTimestampMonotonic=0
ActiveExitTimestampMonotonic=0
InactiveEnterTimestampMonotonic=0
CanStart=no
CanStop=yes
CanReload=no
CanIsolate=no
CanFreeze=no
StopWhenUnneeded=no
RefuseManualStart=no
RefuseManualStop=no
AllowIsolate=no
DefaultDependencies=yes
OnSuccessJobMode=fail
OnFailureJobMode=replace
IgnoreOnIsolate=no
NeedDaemonReload=no
JobTimeoutUSec=infinity
JobRunningTimeoutUSec=infinity
JobTimeoutAction=none
ConditionResult=no
AssertResult=no
ConditionTimestampMonotonic=0
AssertTimestampMonotonic=0
LoadError=org.freedesktop.systemd1.UnitMasked "Unit cryptsetup.target
is masked."
Transient=no
Perpetual=no
StartLimitIntervalUSec=10s
StartLimitBurst=5
StartLimitAction=none
FailureAction=none
SuccessAction=none
CollectMode=inactive
Also, Pasted all the logs in
https://gist.github.com/sumitkumar1-trellix/cefb8477e9e6bde05419f8a0e842994d
Please let me know if any other config/logs required.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20250116/9ed49892/attachment-0001.htm>
More information about the systemd-devel
mailing list