[systemd-devel] Improve boot-time of systemd-based device, revisited

[cee1]

Hi all,

I've recently got another chance to improve the boot-time of a
systemd-based device. I'd like to share the experience here, and some
thoughts and questions.

The first time I tried to improve the boot-time of systemd:
http://lists.freedesktop.org/archives/systemd-devel/2011-March/001707.html,
after that, we have systemd-bootchart and systemd-analyze, which help
a lot.

It seems the biggest challenge of reducing boot-time of the ARM board
at hand is taking are of the poor I/O performance:
* A single fgets() call may randomly cause 200-300ms
* A (big)service may spend 2-3s to complete its so loading - only
~100ms spent on CPU.

I tried to first delay services which are less important, to save the
I/O bandwidth in the early stage, and raise the priority of important
services to SCHED_RR/IOPRIO_CLASS_RT:
1. I need to find the "top I/O hunger" processes (and then delay them
if not important), but it's not straightforward to figure it out in
bootchart, so adding *** iotop feature *** in bootchart seems very
useful.

2. I think raising CPU scheduling priority works because it reduces
chances of issuing I/O requests from other processes. Some thoughts:
* The priority feature of I/O scheduler(CFQ) seems not work very well
- IDLE I/O can still slow down Normal/RT I/O [1]
* I don't know the detail of CFQ, but I wonder whether a "rate limit"
helps - may reduce the latency between issuing I/O command and full
filling the command?

Last, I tried some readahead(ureadahead), but not do the magic, I
guess it's because I/O is busy in the early stage, there's simply no
"ahead" chance.
What readahead helps, IMHO, is a snapshot of accessed disk blocks
during boot up, in the order of they're requested. Thus a linear
readahead against the snapshot will always read ahead of actual
requesting blocks.

BTW, systemd-bootchart has a option to chart entropy, how is the
entropy involved in boot up procedure?



---
1. http://linux-kernel.vger.kernel.narkive.com/0FC8rduf/ioprio-set-idle-class-doesn-t-work-as-its-name-suggests


Regards,

- cee1