[systemd-devel] service fails to use the latest value of the slice

[TARANA, YASHASHVI]

Hi Lennart,

Thank you for getting back. I just subscribed to the systemd-devel mailing list and hence I'm resending the mail.

We were following below article to get around the situation where a process fails to get realtime priority on a system once the CPUAccounting is enabled.
https://access.redhat.com/articles/3696121

It suggests to create your own slice and use it for your service.

We would like to handle a situation where there are 2 such slices (A.slice & B.slice, created by different applications) on the system. 

Sample: test.service
	[Service]
	ExecStart=/root/test start
	ExecStop=/root/test stop
	Type=oneshot
	RemainAfterExit=yes

Here, the binary at ExecStart (test) requires realtime priority.

Scenario 1:
---------------
Steps to Reproduce:
1. Suppose I have 2 slices, A.slice (cpu.rt_runtime_us value = 950000) and B.slice (cpu.rt_runtime_us  value = 0) 2. Currently the service test.service is attached to A.slice 3. Now I changed cpu.rt_runtime_us values for these slices:  A.slice to 0 and B.slice to 950000 (The values of A.slice and B.slice have been exchanged) 4. When I stop and start the service, binary at ExecStart determines that B.slice is the eligible slice and it updates the slice name in the configuration files and in drop-ins.
5. Updated slice information not reflected when start the binary and the start fails to set "realtime" priority.
6. Restart of the same service again works fine and attaches to correct slice (B.slice).

Is this expected behavior? Is there a way to reload the systemd configuration during runtime (i.e during systemctl start <>) ?

Scenario 2:
--------------
	
To work around the above problem in scenario 1 , I followed the below steps after step 3 above.

4. When I stop and start service again, the binary at Execstart determines the right slice( in this case, B.slice) and pushes its pid to B.slice tasks file(/sys/fs/cgroup/cpu,cpuacct/B.slice/tasks) in addition to updating the drop-ins.
5. The test service start successfully now but systemctl status test.service still shows that test.service is attached to the old slice (A.slice), although its pid is part of B.slice now.

I have below queries as part of Scenario 2:
  -  Is there a way to update the slice information for a service at runtime? Or by executing some command ?
 - Will it impact the CPUAccounting feature (which is enabled for both slices), since the pid is part of different slice(B.slice), but service still reflects A.slice?

Regards,
Yashashvi

-----Original Message-----
From: Lennart Poettering [mailto:lennart at poettering.net] 
Sent: Thursday, July 25, 2019 10:45 PM
To: Tiwari, Hari Sahaya <hari-sahaya.tiwari at hpe.com>
Cc: systemd-devel at lists.freedesktop.org; TARANA, YASHASHVI <yashashvi.tarana at hpe.com>
Subject: Re: [systemd-devel] service fails to use the latest value of the slice

On Do, 25.07.19 13:02, Tiwari, Hari Sahaya (hari-sahaya.tiwari at hpe.com) wrote:

> Hello,
> I have one query on the behaviour I am overserving with systemd service.
>
> Below are the contents of service file,
>
> # cat /usr/lib/systemd/system/qs.service
> [Unit]
> Description=init script
> After=network.target
>
> [Service]
> ExecStartPre=/bin/sh /usr/local/cmcluster/bin/realtimeslice.sh -s 
> /usr/lib/systemd/system/qs.service
> ExecStart=/usr/local/qs/bin/qs
> Type=simple
>
> [Install]
> WantedBy=multi-user.target
>
> What realtimeslice.sh does is to identify a slice having RT quantum
> (cpu.rt_runtime_us) as 950000.  Once the slice is identified the 
> service attaches the binary mentioned in ExecStart to that slice.
> This is done because binary(/usr/local/qs/bin/qs) tries to set 
> realtime priority.

I have trouble understanding this. I think there's some misconception about what's systemd's slice concept is about.

Also note: systemd owns the cgroup tree. If you make changes to the cgroup tree you are on your own, you are not supposed to, except if you asked four own delegated subtree.

This means: attaching a process systemd manages for you to a different cgroup with your own code is not supported, you are on your own.

That said, you can let systemd create cgroups and then adjust the attributes that systemd doesn't manager yourself, such as the RT attributes. For that use an ExecStartPre= that just sets these attributes, and maybe make sure With CPUWeight=100 that you actually get properly added to the "cpu" hierarchy...

Lennart

--
Lennart Poettering, Berlin