[systemd-devel] Stricter handling of failing mounts during boot under systemd - crap idea !

[jon]

On Mon, 2015-06-29 at 22:16 +0200, Reindl Harald wrote:
> 
> Am 29.06.2015 um 21:36 schrieb jon:
> > On Mon, 2015-06-29 at 20:50 +0200, Lennart Poettering wrote:
> >> On Mon, 29.06.15 19:20, jon (jon at jonshouse.co.uk) wrote:
> >>
> >>> Reversing the logic by adding a "mustexist" fstab option and keeping the
> >>> default behaviour would fix it.
> >>
> >> At this time, systemd has been working this way for 5y now. The
> >> behaviour it implements is also the right behaviour I am sure, and the
> >> "nofail" switch predates systemd even.
> > I disagree strongly.  As I said the "option" did not do anything... so
> > the change only really happened when systemd coded it. Very people are
> > using systemd, so this change may be "stable old code" in your world, in
> > my world it "new" and its behaviour is "wrong" !
> 
> while i often disagree with systemd developers
> *that* behavior is simply correct
Please justify that !   Correct how ?

The system behaviour to date may not have been the "correct" behaviour,
but the new behaviour is a change, so to my mind it is a "change of
default".

The behaviour now may have been the intended behaviour, but that does
not make it in any way correct! 

If I make a device in red for 10 years, then one day make it green, I
cant then say "I always intended it to be green" and then assume all my
"red" users will be satisfied with that answer !

> 
> >>   Hence I am very sure the
> >> default behaviour should stay the way it is.
> > Your default behaviour or mine !
> >
> > Many people that I know who run linux for real work have been using
> > systemd for 5 mins, most have yet to discover it at all !
> 
> well, it needs much more than 5 minutes to get into a new core part of 
> the system and that said: Fedora users are using systemd now since 2011 
> me including in production
> 
> > The first I knew about is when Debian adopted it, I have been using
> > systemd for a few hours only. It may be your 5 year old pet, but to me
> > it just a new set of problems to solve.
> 
> you can't blame others for that. systemd was available and widely known 
> before
Yes Yes Yes.....

I get bugger all free time. The time I have is spent building things.
Today I building a Rasperry Pi based jig to re-flash a device with
embedded Wifi.  Next week I will be working on an embedded control
system, week after maybe integrating a device with a web front end. I
simply don't get the free time to try new things just in case some
distro maintainer decides that is a good idea, or fashionable or
whatever justification they use for costing me work ;-)

> 
> > I normally install machines with Debian stable,  I am just discovering
> > systemd for the first time.
> 
> and *that* is the problem not a *minor* change, i would understand if 
> you have to change /etc/fstab for 5000 machines but even then: large 
> setups are maintained not by login everywhere manually and make the same 
> change, especially that you can make this change *before* upgrades 
> because it don't harm sysvinit systems, the have no problem with "nofail"
Yes I do follow you logic, but I am not sure that most distros would not
simply warn "unknown mount option bla.."

I have done a few roll outs, I also maintained my own distribution for
several embedded products and wrote a linux network installer to
manufacture devices, I do have some experience. 

As a point of interest the fstab in each embedded device contains an
entry for two FS that doe not exist at initial boot.

The installer for that product uses creates a "/" filesystem with a
complete a pre-configured linux and references two extra filesystems
"/configure" and "/data".  

/configure is created after the machines first run (again in the
factory) and /data is created when the user sets up the machine (a
camera DVR box). /data is also re-created by an mkfs if the user chooses
to reset the box to defaults or replace the internal disk.   

I mention this to explain that referencing FS that may not currently
exist, or existed only for a short time is quite normal in my field. To
my knowledge many products do this internally. I tend therefore to do
similar things with my servers, on the not unreasonable grounds that to
date it has always worked.

> >>> Bringing up networking/sshd in parallel to the admin shell would also
> >>> mitigate my issue....
> >>
> >> That's a distro decision really. Note though that many networking
> >> implementations as well as sshd are actually not ready to run in
> >> early-boot, like the emergecny mode is. i.e. they assume access to
> >> /var works, use PAM, and so on, which you better avoid if you want to
> >> run in that boot phase.
> >
> > Hmmm ... it used to be possible with telnetd, so I suspect it is still
> > possible with sshd.
> 
> not relieable, the emergency shell is even there when mounting the 
> rootfs fails and then you can't bring up most services
Hmmmmm ... emergency shell is correct, ever tried fixing anything from
the initrd util set alone - or just busybox and most of the kernel
modules missing! Most people reach for a re-install or start a full fat
linux from LAN or removable media to repair at that point.


> but i agree that *trying to bring up network and sshd* would be not a 
> bad idea, in case the problem is with a unimportant datadisk it may help
> 
> > This is the "problem" with systemd, by changing one small behaviour it
> > now requires many many changes to get a truly useful system behaviour
> > back.
> 
> honestly it is not normal that mountpoints disappear like in your case 
> and even if - a machine which is that important usually has a *tested* 
> setup and is reachable via KVM or something similar.
Is and "is conveniently" are two VERY different things.

If I review my friends technical setups and companies with linux
machines that I know almost all of them are running a headless server
with no local display and no KVM ! 

Belief that *most* users have easy access to keyboard/display is a data
centre bias.   Most machines I know are on home LANs, or small
industrial units in corners, or in factories under a desk or in a rack
with entirely unrelated hardware (no KVM, no monitor, no keyboard, no
mouse!). They are installed once, pulled out when they fail only.

Out of my 6 technical friends all but one is running at least one linux
server headless with just power and ethernet, some of the people I know
would even have to unplug the machine and move it to another room to get
a display/keyboard on it - so my setup (at least in my peer group here)
is not at all uncommon. 

Here I have 7 servers, and 28 linux based devices, all the servers are
headless. I have one montior/keyboard (no mouse) near the server rack,
but as I mentioned any machine would need the display connected and then
to be power cycled before it would generate any output.

> 
> >> As you might know my company cares about containers, big servers
> >> primarily, while I personally run things on a laptop and a smaller
> >> server on the Internet. Hence believe me that I usually care about
> >> laptop setups at least as much as for server setups.
> > Nope, did not know that, interesting.
> 
> you know Redhat?
I know "of" Redhat, but I have not used it personally since Redhat 9.

> read recent IT news about Redhat and containers
OK, I have never investigated such things. I will read up on the subject
but I suspect containers are something I don't need for my type of
workload.
I (sparingly) use virtual machines for development on my desktop, but
all "live" servers I use are real OS with no virtual environment.

Thanks,
Jon