User bus conclusion

[Lennart Poettering]

On Wed, 10.11.10 19:01, Thiago Macieira (thiago at kde.org) wrote:

> > And it only gets messier from there. As soon as you save state that is
> > dependent on the screen you lost. That's why I believe we shouldn't ever
> > support multiple graphical logins.
> 
> The reason I didn't see a problem is that I don't think the Plasma 
> architecture has that problem. The configuration files are reloaded prior to 
> saving, if I understand correctly what I'm being told in the KDE mailing list, 
> and the change is saved to the config file immediately. That means the changes 
> applied to one screen will not apply to the other until the application is 
> restarted or reloads its configuration.

OK, so I log in twice, arrange my applets, and then log out of both, and
everything I configured on the first login is completely lost? That's
what I call broken.

> > Not entirely. "systemd as a system manager" is a distro decision. "systemd
> > as a session manager" is a decision for the DEs.
> 
> Ok, that's the first I'm hearing of that. I believe we must create a separate 
> discussion on this. Have you engaged anyone from KDE and other DEs to discuss 
> this?

I mentioned this briefly to some KDE folks as OSC. However before we can
start the conversations with the KDE people we needed to make our own
mind up. I think the key folks in the systemd and GNOME scene now have
mostly agreed on something here, at LPC/GS. However only parts of that
are implemented so far, so we don't know whether that'll fly in the end.

> > > The problem being solved by fork+dlopen is the startup time for the
> > > relocation and basic initialisation of libraries (global statics,
> > > mostly, but could also be loading plugins, parsing basic config, etc.),
> > > as well as for sharing read- only but dirty memory.
> > 
> > Yes, I know why KDE does this. Doesn't change the fact that it is a bad
> > idea. But this is nothing to discuss here and now.
> 
> It might be, but it's solving a problem that isn't solved elsewhere.

No, it's working around a problem that should be solved at the right
place.

> > Yes, I know. If loading C++ libs/binaries is slow then maybe it's a good
> > idea to fix that instead of coming up with crazy hacks that break the
> > audit trail, cause constant random page faults in the session manager,
> > leaves mutexes in an undefined state, breaks threading and is actually
> > explicitly recommended not do by the POSIX specs.
> 
> Ouch, that's not what we're doing. The process that forks doesn't have any 
> locked mutexes or even spawned threads. Its only purpose is to preload some 
> libraries so that libdl has a chance to do its work only once (it's also run 
> with LD_BIND_NOW=1). Maybe this is solved with prelinking, but I haven't 
> personally seen much of an uptake of this technology...

Ok, well, Meego does the dlopen() trick too, and they initialize the
libs before forking, sorry for collating that.

Fedora uses prelinking by default. Avoiding relocations entirely instead of
just doing them earlier seems a lot more appropriate.

> What does the session manager (systemd here?) have to do with page faults? Or 
> random even?

Let's say systemd also supported dlopen() style execution. Here's what
would happen:

systemd initializes, loads service data, begins to fork off
processes. These forked off processes became COW copies of the systemd
process, and use dlopen() to jump into some app code. systemd continues
to run, changes a variable here and there and everywhere, since it goes
on with start-up. Now, since a forked app is a COW clone of the parent
every single page the parent processes accesses needs to pagefault. That
is slow and ridiculous since you end up copying memory for no purpose at
all: the child actually doesn't even have the slightest interest in the
memory copied. Now multiply that by every single app you spawn this way.

Hence, basically, systemd will always run into page faults, for every
single page it writes to, a gazillion of times, since we forked of a
gazillion of processes. And all the processes forked off will slowly
duplicate the original systemd memory set in their own processes even
though they don't even have the slightest interest in the actual data.

Or in other words: it's a really bad idea to do this. You definitely
want to decouple the memory layout completely not only by COW, and Unix
has a system call for that. It's called exec().

This is by no means a new problem. In fact the POSIX docs actively
suggest not to spend too much time between fork() and exec() to avoid
this avalanche of page faults in the parent process. And people have
been bitten by this before, for example udev was. 

And it's difficult to track down, since the parent appears to be
misbehaving, not the child.

And of course, the audit trail is seriously deformed by doing apps
startups without exec(). And SELinux labelling goes wrong. 

It's just a bad idea. Fix exec(), fix ld.so. Don't work around it!

> > > Worse, NVidia's libGL.so has impure text (not -fPIC) so each process
> > > using OpenGL on an NVidia system takes up 15 MB of additional, private
> > > memory.
> > 
> > Well, it's sad that KDE is apparently specifically designed for closed
> > source drivers, instead of cleanliness and correctness.
> 
> It's not designing for broken drivers. It's working around the stubbornness of 
> NVidia, by saving our users tens of megabytes of memory used, plus load-time 
> relocations. It only takes 7 processes that need to do OpenGL to consume 100 
> MB of memory. libnvidia-glcore.so.256.44 has 395613 relative relocations out 
> of 402050 total relocations.

Well, however that may be. On GNOME we actually try to do things
properly. Not just work-arounds.

Lennart

-- 
Lennart Poettering - Red Hat, Inc.