[systemd-devel] Slow startup of systemd-journal on BTRFS

[Goffredo Baroncelli]

On 06/14/2014 12:59 PM, Kai Krakow wrote:
[...]
> 
> I think that systemd is even one of the early supporters of btrfs because it 
> will defragment readahead files on boot from btrfs. 

In know that systemd does readahead, but it is the first time that I heard that it does defrag too. Could you elaborate ?

> I'd suggest the problem 
> is to be found in the different semantics with COW filesystems. 
[--cut--cut--cut--]

The problem is due to the interaction between BTRFS and systemd. Both behave differently regarding their predecessor. 

[--cut--cut--cut--]

> So let's start with my journals, on btrfs:
> 
> $ sudo filefrag *
> system at 0004fad12dae7676-98627a3d7df4e35e.journal~: 2 extents found
> system at 0004fae8ea4b84a4-3a2dc4a93c5f7dc9.journal~: 2 extents found
> system at 806cd49faa074a49b6cde5ff6fca8adc-000000000008e4cc-0004f82580cdcb45.journal: 
> 5 extents found
> system at 806cd49faa074a49b6cde5ff6fca8adc-0000000000097959-0004f89c2e8aff87.journal: 
> 5 extents found                                                                                                                  
> system at 806cd49faa074a49b6cde5ff6fca8adc-00000000000a166d-0004f98d7e04157c.journal: 
> 5 extents found                                                                                                                  
> system at 806cd49faa074a49b6cde5ff6fca8adc-00000000000aad59-0004fa379b9a1fdf.journal: 
> 5 extents found
> system at ec16f60db38f43619f8337153a1cc024-0000000000000001-0004fae8e5057259.journal: 
> 5 extents found
> system at ec16f60db38f43619f8337153a1cc024-00000000000092b1-0004fb59b1d034ad.journal: 
> 5 extents found
> system.journal: 9 extents found
> user-500 at e4209c6628ed4a65954678b8011ad73f-0000000000085b7a-0004f77d25ebba04.journal: 
> 2 extents found
> user-500 at e4209c6628ed4a65954678b8011ad73f-000000000008e7fb-0004f83c7bf18294.journal: 
> 2 extents found
> user-500 at e4209c6628ed4a65954678b8011ad73f-0000000000097fe4-0004f8ae69c198ca.journal: 
> 2 extents found
> user-500 at e4209c6628ed4a65954678b8011ad73f-00000000000a1a7e-0004f9966e9c69d8.journal: 
> 2 extents found
> user-500.journal: 2 extents found
> 
> I don't think these are too bad values, eh?
> 
> Well, how did I accomblish that?
> 
> First, I've set the journal directories nocow. 

If you use nocow, you lost the btrfs ability to rebuild a RAID array discarding the wrong sector. With the systemd journal checksum, you can say that a data is wrong, but BTRFS with its checksum (when used in raid mode) is able to *correct* the data too.

[...]


> 
> Back to the extents counts: What I did next was implementing a defrag job 
> that regularly defrags the journal (actually, the complete log directory as 
> other log files suffer the same problem):
> 
> $ cat /usr/local/sbin/defrag-logs.rb 
> #!/bin/sh
> exec btrfs filesystem defragment -czlib -r /var/log

I think that this is a more viable solution; but what seems to me strange is the fact that the fragmentation level of my rsyslog file is a lot lower.

[...]


-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5