[systemd-devel] systemd failing to close unwanted file descriptors & FDS spawning and crashing

Christopher Obbard chris.obbard at collabora.com
Fri Mar 4 09:26:37 UTC 2022


Hi Mantas,

On 03/03/2022 20:30, Mantas Mikulėnas wrote:
> Ah, right, I forgot – since this is done in the service child (right 
> before exec) and not in the main process, you probably need to add the 
> -f option to make strace follow forks...
> 
> On Thu, Mar 3, 2022, 22:08 Christopher Obbard 
> <chris.obbard at collabora.com <mailto:chris.obbard at collabora.com>> wrote:
> 
>     Hi Mantas,
> 
>     On 03/03/2022 19:18, Mantas Mikulėnas wrote:
>      > On Thu, Mar 3, 2022 at 9:09 PM Christopher Obbard
>      > <chris.obbard at collabora.com <mailto:chris.obbard at collabora.com>
>     <mailto:chris.obbard at collabora.com
>     <mailto:chris.obbard at collabora.com>>> wrote:
>      >
>      >     Hi systemd experts!
>      >
>      >     I am using systemd-247 and systemd-250 on debian system, which is
>      >     running a minimal downstream 5.4 kernel for a Qualcomm board.
>      >
>      >     systemd 241 in debian buster works fine, but systemd 247 (debian
>      >     bullseye) and systemd 250 (debian unstable) seem to get upset
>     about
>      >     file
>      >     descriptors on services. These errors are consistant and the
>     board
>      >     boots
>      >     just fine with init=/bin/sh
>      >
>      >     I've got the required kernel config from README in my kernel,
>     I am
>      >     using
>      >     a heavily patched downstream kernel, but from the following
>     log can you
>      >     suggest anything I can do to debug this (other than throwing
>     the board
>      >     out of the window) ?
>      >
>      >
>      >  From the message, it looks like the error is returned by
>      > close_all_fds() in src/basic/fd-util.c, where the only major
>     change is
>      > that it has been ported to call close_range() if that's available...
>      >
>      > I would boot with init=/bin/sh, then run `exec strace -D -o
>      > /var/log/systemd.trace /lib/systemd/systemd` to get a trace, and
>     see if
>      > the EINVAL actually comes from calling close_range() or from
>     something else.
>      >
>      > --
>      > Mantas Mikulėnas
> 
>     Thanks for your reply. It reproduced nicely with the command you gave.
> 
>     Seems like nothing related to close_range returning EINVAL, only the
>     following calls returned EINVAL:
> 
>     cat systemd-failing.trace | grep EINVAL
>     prctl(PR_CAPBSET_READ, 0x30 /* CAP_??? */) = -1 EINVAL (Invalid
>     argument)
>     prctl(PR_CAPBSET_READ, CAP_CHECKPOINT_RESTORE) = -1 EINVAL (Invalid
>     argument)
>     prctl(PR_CAPBSET_READ, CAP_PERFMON)     = -1 EINVAL (Invalid argument)
>     read(4, 0x55675a75b0, 4095)             = -1 EINVAL (Invalid argument)
>     mount("cgroup2", "/proc/self/fd/4", "cgroup2",
>     MS_NOSUID|MS_NODEV|MS_NOEXEC, "nsdelegate,memory_recursiveprot") = -1
>     EINVAL (Invalid argument)
> 
> 
>     I have attached the full strace output, in case that would be useful?
> 
>     Thanks
>     Chris
> 

Right, so it looks like the call to close_range fails. This is a 5.4 
kernel which doesn;t have close_range - so this is understandable.

For a quick fix, I set have_close_range to false - see the patch 
attached. It seemed to work well.

Since my 5.4 kernel is a heavily modified downstream one - next I will 
check if that syscall was implemented by someone else, and also I will 
check if vanilla systemd works on vanilla 5.4 (there is no reason why it 
shouldn't, right?).


$ cat /mnt/systemd.trace  | grep EINVAL
1     prctl(PR_CAPBSET_READ, 0x30 /* CAP_??? */) = -1 EINVAL (Invalid 
argument)
1     prctl(PR_CAPBSET_READ, CAP_CHECKPOINT_RESTORE) = -1 EINVAL 
(Invalid argument)
1     prctl(PR_CAPBSET_READ, CAP_PERFMON) = -1 EINVAL (Invalid argument)
1     read(4, 0x556968b5b0, 4095)       = -1 EINVAL (Invalid argument)
1     mount("cgroup2", "/proc/self/fd/4", "cgroup2", 
MS_NOSUID|MS_NODEV|MS_NOEXEC, "nsdelegate,memory_recursiveprot") = -1 
EINVAL (Invalid argument)
364   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
364   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
365   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
364   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
365   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
366   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
366   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
365   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
361   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
366   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
370   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
368   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
361   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
370   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
368   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
361   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
370   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
368   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
367   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
362   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
367   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
362   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
367   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
362   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
363   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
360   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
363   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
360   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
363   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
360   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
369   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
369   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
369   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
359   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
359   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
359   <... prctl resumed>)              = -1 EINVAL (Invalid argument)
364   <... ioctl resumed>, 0)           = -1 EINVAL (Invalid argument)
373   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
374   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
375   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
376   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
377   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
378   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
379   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
380   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
381   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
382   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
384   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
385   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
386   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
387   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
388   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
389   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
391   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
392   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
393   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
395   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
397   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
398   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
399   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
400   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
401   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
402   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
403   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
404   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
405   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
406   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
407   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
408   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
409   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
410   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
411   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
412   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
413   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
414   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
415   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
416   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
418   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
419   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
420   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
421   <... close_range resumed>)        = -1 EINVAL (Invalid argument)
422   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
423   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
424   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
417   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)
425   close_range(3, 17, 0)             = -1 EINVAL (Invalid argument)



Thanks
Chris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: HACK-Do-not-assume-we-have-close_range-available.patch
Type: text/x-patch
Size: 817 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/systemd-devel/attachments/20220304/92e0435b/attachment.bin>


More information about the systemd-devel mailing list