<div dir="ltr">Hi everyone,<div><br></div><div>[Disclaimer: cross posting from <a href="https://github.com/containers/podman/discussions/14538">https://github.com/containers/podman/discussions/14538</a>]</div><div><br></div><div><p dir="auto" style="box-sizing:border-box;margin-bottom:16px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";margin-top:0px">Apologies that this is more of a Linux cgroup question than specific to systemd, but I was wondering if someone here might be able to enlighten me...</p><p dir="auto" style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji"">Two questions:</p><ul dir="auto" style="box-sizing:border-box;padding-left:2em;margin-top:0px;margin-bottom:16px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji""><li style="box-sizing:border-box;margin-left:0px">Why on cgroups v1 do the cpuset controller's <code class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0.2em 0.4em;margin:0px;border-radius:6px">cpuset.effective_{cpus,mems}</code> seem to simply not work?</li><li style="box-sizing:border-box;margin-top:0.25em;margin-left:0px">Is there any way to check effective cgroup memory or hugetlb limits? (cgroups v1 or v2)</li></ul><h3 dir="auto" style="box-sizing:border-box;margin-top:24px;margin-bottom:16px;line-height:1.25;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji""><font size="2">Cpuset effective limits</font></h3><div class="gmail-snippet-clipboard-content gmail-notranslate gmail-position-relative gmail-overflow-auto" style="box-sizing:border-box;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";overflow:auto"><pre class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;margin-top:0px;margin-bottom:16px;padding:16px;overflow:auto;line-height:1.45;border-radius:6px"><code style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0px;margin:0px;background:transparent;border-radius:6px;word-break:normal;border:0px;display:inline;overflow:visible;line-height:inherit">root@ubuntu:~# podman run --rm -it --privileged -w /sys/fs/cgroup fedora [root@7b9b67c7e1d4 cgroup]# mkdir cpuset/my-group [root@7b9b67c7e1d4 cgroup]# cat cpuset/cpuset.cpus 0-5 [root@7b9b67c7e1d4 cgroup]# cat cpuset/my-group/cpuset.cpus [root@7b9b67c7e1d4 cgroup]# cat cpuset/my-group/cpuset.effective_cpus </code></pre></div><p dir="auto" style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji"">Expected <code class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0.2em 0.4em;margin:0px;border-radius:6px">cpuset/my-group/cpuset.effective_cpus</code> to give <code class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0.2em 0.4em;margin:0px;border-radius:6px">0-5</code> as set in the parent cgroup. Works as expected on cgroups v2.</p><div class="gmail-snippet-clipboard-content gmail-notranslate gmail-position-relative gmail-overflow-auto" style="box-sizing:border-box;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";overflow:auto"><pre class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;margin-top:0px;margin-bottom:16px;padding:16px;overflow:auto;line-height:1.45;border-radius:6px"><code style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0px;margin:0px;background:transparent;border-radius:6px;word-break:normal;border:0px;display:inline;overflow:visible;line-height:inherit">[root@7b9b67c7e1d4 cgroup]# echo 0-5 > cpuset/my-group/cpuset.cpus [root@7b9b67c7e1d4 cgroup]# cat cpuset/my-group/cpuset.{effective_,}cpus 0-5 0-5 [root@7b9b67c7e1d4 cgroup]# echo 0-4 > cpuset/cpuset.cpus bash: echo: write error: Device or resource busy </code></pre></div><p dir="auto" style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji"">Didn't expect this to fail - shouldn't it automatically impose a stricter limit on any child cgroups? Do I need to manually update all child cgroups first?</p><div class="gmail-snippet-clipboard-content gmail-notranslate gmail-position-relative gmail-overflow-auto" style="box-sizing:border-box;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";overflow:auto"><pre class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;margin-top:0px;margin-bottom:16px;padding:16px;overflow:auto;line-height:1.45;border-radius:6px"><code style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0px;margin:0px;background:transparent;border-radius:6px;word-break:normal;border:0px;display:inline;overflow:visible;line-height:inherit">[root@7b9b67c7e1d4 cgroup]# echo 0-4 > cpuset/my-group/cpuset.cpus [root@7b9b67c7e1d4 cgroup]# cat cpuset/my-group/cpuset.{effective_,}cpus 0-4 0-4 </code></pre></div><p dir="auto" style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji"">Can impose a stricter limit on child cgroups, as expected.</p><div class="gmail-snippet-clipboard-content gmail-notranslate gmail-position-relative gmail-overflow-auto" style="box-sizing:border-box;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";overflow:auto"><pre class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;margin-top:0px;margin-bottom:16px;padding:16px;overflow:auto;line-height:1.45;border-radius:6px"><code style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0px;margin:0px;background:transparent;border-radius:6px;word-break:normal;border:0px;display:inline;overflow:visible;line-height:inherit">[root@7b9b67c7e1d4 cgroup]# echo 0-4 > cpuset/cpuset.cpus [root@7b9b67c7e1d4 cgroup]# echo 0-5 > cpuset/my-group/cpuset.cpus bash: echo: write error: Permission denied </code></pre></div><p dir="auto" style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji"">But can't relax the child's cgroup restriction (i.e. need awareness of CPU restrictions already imposed above - how are you supposed to check this in a private cgroup namespace?).</p><h3 dir="auto" style="box-sizing:border-box;margin-top:24px;margin-bottom:16px;line-height:1.25;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji""><font size="2">Memory/Hugetlb effective limits</font></h3><p dir="auto" style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji"">On cgroups v1:</p><div class="gmail-snippet-clipboard-content gmail-notranslate gmail-position-relative gmail-overflow-auto" style="box-sizing:border-box;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";overflow:auto"><pre class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;margin-top:0px;margin-bottom:16px;padding:16px;overflow:auto;line-height:1.45;border-radius:6px"><code style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0px;margin:0px;background:transparent;border-radius:6px;word-break:normal;border:0px;display:inline;overflow:visible;line-height:inherit">[root@7b9b67c7e1d4 cgroup]# ls memory/ cgroup.clone_children memory.kmem.failcnt memory.kmem.tcp.limit_in_bytes memory.max_usage_in_bytes memory.soft_limit_in_bytes notify_on_release cgroup.event_control memory.kmem.limit_in_bytes memory.kmem.tcp.max_usage_in_bytes memory.move_charge_at_immigrate memory.stat tasks cgroup.procs memory.kmem.max_usage_in_bytes memory.kmem.tcp.usage_in_bytes memory.numa_stat memory.swappiness memory.failcnt memory.kmem.slabinfo memory.kmem.usage_in_bytes memory.oom_control memory.usage_in_bytes memory.force_empty memory.kmem.tcp.failcnt memory.limit_in_bytes memory.pressure_level memory.use_hierarchy </code></pre></div><p dir="auto" style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji"">There is a <code class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0.2em 0.4em;margin:0px;border-radius:6px">memory.limit_in_bytes</code> file, but no <code class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0.2em 0.4em;margin:0px;border-radius:6px">memory.effective_limit_in_bytes</code> to reflect parent cgroup restrictions.</p><p dir="auto" style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji"">Similarly on cgroups v2:</p><div class="gmail-snippet-clipboard-content gmail-notranslate gmail-position-relative gmail-overflow-auto" style="box-sizing:border-box;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";overflow:auto"><pre class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;margin-top:0px;margin-bottom:16px;padding:16px;overflow:auto;line-height:1.45;border-radius:6px"><code style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0px;margin:0px;background:transparent;border-radius:6px;word-break:normal;border:0px;display:inline;overflow:visible;line-height:inherit">[root@0c0d71230663 cgroup]# ls memory.* memory.current memory.events.local memory.low memory.min memory.oom.group memory.stat memory.swap.events memory.swap.max memory.events memory.high memory.max memory.numa_stat memory.pressure memory.swap.current memory.swap.high </code></pre></div><p dir="auto" style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji"">There is a <code class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0.2em 0.4em;margin:0px;border-radius:6px">memory.max</code> file, but not <code class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0.2em 0.4em;margin:0px;border-radius:6px">memory.max.effective</code> (corresponding to <code class="gmail-notranslate" style="box-sizing:border-box;font-family:ui-monospace,SFMono-Regular,"SF Mono",Menlo,Consolas,"Liberation Mono",monospace;padding:0.2em 0.4em;margin:0px;border-radius:6px">cpuset.cpus.effective</code>).</p><p dir="auto" style="box-sizing:border-box;margin-top:0px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";margin-bottom:0px">I guess you could traverse up the cgroup hierarchy to find the smallest limit being imposed... But this isn't possible inside a private cgroup namespace. Is there any way to find the actual cgroup limit imposed?</p><p dir="auto" style="box-sizing:border-box;margin-top:0px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";margin-bottom:0px"><br></p><p dir="auto" style="box-sizing:border-box;margin-top:0px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";margin-bottom:0px"><br></p><p style="box-sizing:border-box;margin-top:0px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";margin-bottom:0px">Any insights welcome!</p><p style="box-sizing:border-box;margin-top:0px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";margin-bottom:0px"><br></p><p style="box-sizing:border-box;margin-top:0px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";margin-bottom:0px">Thanks,</p><p style="box-sizing:border-box;margin-top:0px;color:rgb(36,41,47);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";margin-bottom:0px">Lewis</p></div></div>