[systemd-devel] Setting Environment Configuration (Affinity) for Slices

[Chris Bell]

On 2015-10-19 16:54, Lennart Poettering wrote:
> On Mon, 19.10.15 11:58, Chris Bell (cwbell at narmos.org) wrote:
> 
>> Is there a way to set an affinity for an entire slice?
> 
> No there is not. But it's definitely our intention to add this, and
> expose the "cpuset" cgroup controller this way. Unfortunately the
> "cpuset" controller of the kernel currently exposes really awful
> behaviour, hence we have not exposed its functionality this way.
> 
> However, I just had a long chat with Tejun Heo about this, and we came
> to the conclusion that's probably safe to expose a minimal subset of
> cpuset now, and reuse the existing CPUAffinity= service setting for
> that: right now, it only affects the main process of a service at fork
> time (and all child processes forked off from that, recursively), by
> using sched_setaffinity(). Our idea would be to propagate it into the
> "cpuset.cpus" field too, so that the setting is first passed to
> sched_setaffinity(), and then also written to the cpuset
> hierarchy. This should be pretty safe, and allow us to make this
> available in slices too. It would result in a slight change of
> behaviour though, as making adjustments to cpuset would mean that
> daemons cannot extend their affinity with sched_setaffinity() above
> what was set with cpuset anymore. But I think this is OK.

So, there's a good chance for a subset cpuset-related options at the 
slice level relatively soon, but full capabilities will have to wait 
until kernel cgroups are improved?

>> Is there any way I >> can set an environment config directive for
>> an entire slice? Or do I need to do it per-service?
> 
> The latter. Slices are really about resource control, and an env var
> really isn't a resource control knob.
> 
>> Alternatively, is there a way (and this sounds way too hacky) to
>> hierarchically order a slice under a service?
> 
> Nope. Slices are the inner nodes of the resource control tree, and
> services/scopes are the leaves. That's how they are defined.
> 
>> I've noticed that systemd-nspawn@<machine>.servce has a child 
>> 'system.slice'
>> though I don't know if that setup can enforce what I'd like it to.
> 
> Well, thats because nspawn is a delegation unit that encapsulates a
> completely new cgroup hierarchy of its own, managed by a new systemd
> instance.

Aha, that makes sense.

> I am not sure I understand what you want to to do with the env vars
> precisely? what kind of env vars do you intend to set?

Basically, I have a number of services that may or may not be running at 
any given time, based on the whims of the users. All of these services 
are hosted services of some type, and occasionally they have been known 
to eat all CPU cores, lagging everything else. I'm working on setting up 
CPU shares and other resource controls to try and keep resources 
available for immediate execution of system processes, services, etc. 
I'd prefer to do this with affinity; assign critical processes to CPUs 
0-1, and the rest limited to subsets of the available remaining CPUs. I 
was hoping I could do this in one run by saying "everything in this 
slice can must run with this affinity." I can do it on a per-service 
basis, but with a large number of services it gets tedious.

I also think it would be convenient in some cases to be able to use the 
'Nice' and 'Private{Network,Devices,etc}' directives apply to an entire 
slice. That way I can use slices to control, manage, and group related 
services. (Example: I'd like to manage postfix and dovecot together in 
system-mail.slice. I'd like to be able to use the slice to set exec 
options for both services. Then if I add another service to 
system-mail.slice, it would also automatically be constrained by the 
limits set in system-mail.slice.)

Basically, I think this would be a useful hierarchy level for more 
coarse-grained service group management and configuration.

--Chris