[systemd-devel] fully volatile running from ram

[Lennart Poettering]

On Thu, 27.04.17 12:52, jr (darwinskernel at gmail.com) wrote:

> On Wed, Apr 26, 2017 at 04:08:21PM +0200, Lennart Poettering wrote:
> > On Tue, 25.04.17 13:13, jr (darwinskernel at gmail.com) wrote:
> > 
> > > hello,
> > > 
> > > in a fully-volatile boot scenario /usr from a physical disk gets mounted on
> > > top of an instance of a tmpfs. my first question is why is that necessary?
> > > (the tmpfs part i mean)
> > 
> > I am not sure I grok what you are trying to say? tmpfs is how on Linux
> > you can easily have a volatile file system, as it lives entirely in
> > memory and never hits the disk (admittedly modulo swapping).
> > 
> 
> but once you mount over that tmpfs from disk the overlaying fs will hide
> the underlying tmpfs, no? baring the fs-caching done in kernel
> (readahead?), for anything that needs to be loaded into memory for
> executing or otherwise disk has to be touched for, well reading, at least
> once, no? this is the part i'm trying to understand, if the overlaying fs
> is mounted from physical disk, does mounting over tmpfs causes kernel to
> cache that fs entirely into memory?
> 
> this looks a lot like initramfs where it is an instance of tmpfs but that
> also gets *shadowed* once "real_root" is switched to, no?

I still don't grok what you are trying to do, but do note that open
files (regardless whether referenced via an open fd, or via an mmap()
mapping, for example because of a ELF binary being mapped into memory)
will always continue to reference the original files on the original
file systems they were opened from regardless if the file system is
now obstructed and the paths originally used to open them would refer
to a different file or file system now.

> > > my second question is, would it be possible to do the same but rather
> > > than mounting the /usr *populate* the said tmpfs with OS tree from said
> > > physical disk, preferably in a blocked or fs cached setup (db-cache or
> > > bcachefs). i realise that this can be done easily in initrd or even
> > > initramfs can hold the /usr but the problem there is when we boot
> > > "developmen" and not "production" in which case we want updates to be
> > > written to disk.
> > 
> > I am not grokking this question either, but keeping the whole OS in
> > memory of course means you need a lot of memory.  Moreover depending on
> > disk speeds it means your boot will be quite slow, as you copy stuff over
> > every single time before you can boot.  If you copy things around like
> > that during boot the best thing would be to do it in the initrd, as
> > otherwise you are running the system already, that you are about to
> > prepare for running, and dropping open references the the old files is
> > hard (though not entirely impossible, after all there is "systemctl
> > daemon-reexec" and things).
> 
> no, no, i'm thinking systemd as rdinit rather than init; i.e. initramfs is
> the real_root. one way of doing it is to pack the /usr into a initramfs
> archive and either build it into kernel or pass it via bootload (never
> worked for me)? then you boot the system into ram, enjoy blazing fast
> responsiveness of it but there comes along some update that one would like
> to apply and it turns out be really great that update. but how do you make
> it stick then? in fact if /var is not volatile and your package manager
> keeps it's records there (in my case portage does) on the next boot system
> is confused because it thinks that updates are going to be there. this has
> a number of solutions; bcachefs or dm-cache or even --overlay.
> --overlay is cool since it stages the upgrade; caching solutions are for
> performance though. i just don't know if the hooks are there (kernel
> and/or systmd) too boot the system this way? i.e. populate the initramfs
> from current or next or the one-after gold-master which resides on disk;
> start working on initramfs; associate this initramfs with original or
> another block-device or subvol of an fs on the disk and let our chosen
> caching system take care of mirroring our working tree with said *backup*. 
> next reboot we should have the option of roll-back or continuing with our
> work and so on.
> 
> please, please let me know if i'm still making no-sense :) English is not
> my strong suit and on top of that i'm horrible at explaining something.

Still not grokking what you are trying to do, but do note that an
initrd is ultimately just a tmpfs preinitialized from a cpio archive.

Also note that if you have a directory tree on disk there's little
reason to copy it into tmpfs performance-wise. The kernel buffer cache
will cache the data on disk into RAM anyway as it is being accessed.

Lennart

-- 
Lennart Poettering, Red Hat