<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><a name="_MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks for this and the other suggestions!<o:p></o:p></span></a></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">So for starters we’ll disable logind and dbus, increase watchdogsec and see where the footprint is – before disabling journald if necessary in a next step.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Regards<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Michael<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Mantas Mikulėnas [mailto:grawity@gmail.com]
<br>
<b>Sent:</b> Wednesday, June 08, 2016 11:35 AM<br>
<b>To:</b> Hebenstreit, Michael<br>
<b>Cc:</b> Systemd<br>
<b>Subject:</b> Re: [systemd-devel] question on special configuration case<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">This sounds like you could start by unsetting WatchdogSec= for those daemons. Other than the watchdog, they shouldn't be using any CPU unless explicitly contacted.<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">On Wed, Jun 8, 2016, 02:50 Hebenstreit, Michael <<a href="mailto:michael.hebenstreit@intel.com">michael.hebenstreit@intel.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">The base system is actually pretty large (currently 1200 packages) - I hate that myself. Still performance wise the packages are not the issue. The SSDs used can easily handle that, and library loads are only happening once at startup (where
the difference van be measured, but if the runtime is 24h startup time of 1s are not an issue). Kernel is tweaked, but those changes are relatively small.<br>
<br>
The single problem biggest problem is OS noise. Aka every cycle that the CPU(s) are working on anything but the application. This is caused by a combination of "large number of nodes" and "tightly coupled job processes".<br>
<br>
Our current (RH6) based system runs with a minimal number of demons, none of them taking up any CPU time unless they are used. Systemd process are not so well behaved. After a few hours of running they are already at a few seconds. On a single system - or systems
working independent like server farms - that is not an issue. On our systems each second lost is multiplied by the number of nodes in the jobs (let's say 200, but it could also be up to 10000 or more on large installations) due to tight coupling. If 3 demons
use 1s a day each (and this is realistic on Xeon Phi Knights Landing systems), that's slowing down the performance by almost 1% (3 * 200 / 86400 = 0.7% to be exact). And - we do not gain anything from those demons after initial startup!<br>
<br>
My worst experience with such issues was on a cluster that lost 20% application performance due to a badly configured crond demon. Now I do not expect systemd to have such a negative impact, but even 1%, or even 0.5% of expected loss are too much in our case.<br>
<br>
<o:p></o:p></p>
</blockquote>
</div>
</div>
</body>
</html>