[systemd-devel] question on special configuration case

[Hebenstreit, Michael]

> Really?  No journal messages are getting created at all?  No users logging in/out?  What does strace show on those processes?

Yes, messages are created - but I'm not interested in them. Maybe a user logs in for a 6h job - that's already tracked by the cluster software. There are virtually no demons running, no changes to the hardware - so all those demons are doing are looking out for themselves. Not really productive

> So you are hurting all 253 cores because you can't spare 1?

Situation is a bit more complex. I have 64 physical cores, with 4 units each for integer operations and 2 floating point units. So essentially if I reserve one integer unit for the OS, due to cache hierarchies and other oddities, I essentially take down 4 cores. The applications typically scale best if they run on a power of 2 number of cores. 

> Again, that's not the issue, you can't see the time the kernel is using to do its work, but it is there (interrupts, scheduling, housekeeping,
etc.)

shouldn't that show up in the time for worker threads? And I'm not arguing you are wrong. We should minimize that and if possible keep all OS on an extra core. That does not make my argument invalid those demons are doing nothing more than housekeeping themselves in a very complicated fashion and they are wasting resources. 



-----Original Message-----
From: Greg KH [mailto:gregkh at linuxfoundation.org] 
Sent: Wednesday, June 08, 2016 11:01 AM
To: Hebenstreit, Michael
Cc: Jóhann B. Guðmundsson; Lennart Poettering; systemd-devel at lists.freedesktop.org
Subject: Re: [systemd-devel] question on special configuration case

On Wed, Jun 08, 2016 at 02:04:48AM +0000, Hebenstreit, Michael wrote:
> > What processes are showing up in your count?  Perhaps it's just a 
> > bug that needs to be fixed.
> /bin/dbus-daemon
> /usr/lib/systemd/systemd-journald
> /usr/lib/systemd/systemd-logind
> 
> I understand from the previous mails those are necessary to make 
> systemd work - but here they are doing nothing more than talking to 
> each other!

Really?  No journal messages are getting created at all?  No users logging in/out?  What does strace show on those processes?

> > That what "most" other system designers in your situation do :)
> Unfortunately I cannot reserve a CPU for OS - I'd like to, but the app 
> developers insist to use all 254 cores available

So you are hurting all 253 cores because you can't spare 1?  If you do the math I think you will find you will get increased throughput.  But what do I know... :)

> > Your kernel is eating more CPU time than those 1s numbers, why you 
> > aren't complaining about that seems strange to me :)
> I also check kernel - last time I look on RH6 all kernel threads 
> taking up clock ticks were actually doing work ^^ No time yet to do 
> the same on RH7 kernel

Again, that's not the issue, you can't see the time the kernel is using to do its work, but it is there (interrupts, scheduling, housekeeping,
etc.)  So get it out of the way entirely and see how much faster your application runs without it even present on those cpus, if you really have cpu bound processes.  That's what the feature was made for, people in your situation, to ignore it and try to go after something else seems very strange to me.

greg k-h