[systemd-devel] Setting Environment Configuration (Affinity) for Slices

[Lennart Poettering]

On Mon, 19.10.15 17:16, Chris Bell (cwbell at narmos.org) wrote:

> >However, I just had a long chat with Tejun Heo about this, and we came
> >to the conclusion that's probably safe to expose a minimal subset of
> >cpuset now, and reuse the existing CPUAffinity= service setting for
> >that: right now, it only affects the main process of a service at fork
> >time (and all child processes forked off from that, recursively), by
> >using sched_setaffinity(). Our idea would be to propagate it into the
> >"cpuset.cpus" field too, so that the setting is first passed to
> >sched_setaffinity(), and then also written to the cpuset
> >hierarchy. This should be pretty safe, and allow us to make this
> >available in slices too. It would result in a slight change of
> >behaviour though, as making adjustments to cpuset would mean that
> >daemons cannot extend their affinity with sched_setaffinity() above
> >what was set with cpuset anymore. But I think this is OK.
> 
> So, there's a good chance for a subset cpuset-related options at the slice
> level relatively soon, but full capabilities will have to wait until kernel
> cgroups are improved?

Well, I am not sure what "full capabilities" really mean here. Much of
the cpuset functionality appears to be little else than just help for
writing shell scripts. That part is certainly nothign we want to
expose.

The other part is NUMA memory node stuff, but supposedly that's stuff
that should be dealt with automatically by the kernel, and not need
user configuration. Hence it's nothign we really want to expose right
anytime soon.

> >I am not sure I understand what you want to to do with the env vars
> >precisely? what kind of env vars do you intend to set?
> 
> Basically, I have a number of services that may or may not be running at any
> given time, based on the whims of the users. All of these services are
> hosted services of some type, and occasionally they have been known to eat
> all CPU cores, lagging everything else. I'm working on setting up CPU shares
> and other resource controls to try and keep resources available for
> immediate execution of system processes, services, etc. I'd prefer to do
> this with affinity; assign critical processes to CPUs 0-1, and the rest
> limited to subsets of the available remaining CPUs. I was hoping I could do
> this in one run by saying "everything in this slice can must run with this
> affinity." I can do it on a per-service basis, but with a large number of
> services it gets tedious.

Well, sure, exposing the cpuset knobs as discussed above should make
this easy, and that's precisely what slices have been introduced for. 

I was mostly wondering about the env var issue you raised...

> I also think it would be convenient in some cases to be able to use the
> 'Nice' and 'Private{Network,Devices,etc}' directives apply to an entire
> slice. That way I can use slices to control, manage, and group related
> services. (Example: I'd like to manage postfix and dovecot together in
> system-mail.slice. I'd like to be able to use the slice to set exec options
> for both services. Then if I add another service to system-mail.slice, it
> would also automatically be constrained by the limits set in
> system-mail.slice.)

Use CPUShares= as per-slice/per-service/per-scope equivalent of
Nice=.

PrivateXYZ= otoh is very specific to what a daemon does, it's a
sandboxing feature, and sandboxes must always be adjusted to the
individual daemons. I doubt that this is something to support as
anything but a service-specific knob.

Lennart

-- 
Lennart Poettering, Red Hat