[systemd-devel] journald leaking maps?

[Colin Guthrie]

'Twas brillig, and Colin Guthrie at 16/10/12 01:06 did gyre and gimble:
> 'Twas brillig, and Lennart Poettering at 16/10/12 00:27 did gyre and gimble:
>> On Thu, 11.10.12 23:31, Colin Guthrie (gmane at colin.guthr.ie) wrote:
>>
>>> Hi,
>>>
>>> This has been discussion on IRC but I've not heard anything about it
>>> specifically nor have I seen any commits relating to it.
>>>
>>> Using journald without persistent logs (i.e. /run only), I see the
>>> memory footprint of journald increasing over time.
>>>
>>> Checking the maps shows a very high count:
>>>
>>> [root at jimmy ~]# cat /proc/$(pidof systemd-journald)/maps | grep
>>> /run/log/journal/6cb2a4b2bd6df042e57da8a4000001d4/system.journal | wc -l
>>> 1641
>>
>> That indeed looks like a lot.
>>
>>> Something is obviously not good there! journald is using something in
>>> the region of 250MB res.
>>>
>>> What's the best way to debug this?
>>
>> If maps are leaked it would be good to figure out of what. So could you
>> go through /proc/$PID/maps and check for a) files with the "(deleted)"
>> suffix, and b) for map for the same range of the same file?
>>
>> If you find a) then there's probably something borked with releasing
>> mmap()s when rotating. If we you find b) then we are apparently too
>> stupid to reuse existing mmaps (which is the whole point of
>> mmap-cache.c where all this code resides...).
> 
> I've attached the map.txt file. I only have one journal file so it's not
> rotated/deleted or anything.
> 
> Can't see any overlaps and with the script below.
> 
> OLDRANGE=; for RANGE in $(cat maps.txt | cut -d' ' -f1); do A=$(echo
> $RANGE | cut -d'-' -f1); if [ -n "$OLDRANGE" ]; then if [ "$A" != "$B"
> ]; then echo "FAIL $OLDRANGE $RANGE"; fi; fi; B=$(echo $RANGE | cut
> -d'-' -f2); OLDRANGE=$RANGE; done
> 
> 
> FAIL 7f45518d8000-7f455196c000 7f4551bb5000-7f4551c48000
> FAIL 7f4551bb5000-7f4551c48000 7f455211b000-7f455211d000
> 
> 
> 
> So this just identifies a few lines at the end which has gaps in the
> mmapping apparently, but no overlaps.
> 
> Does this tell you anything?

OK, I think I see the problem.

Essentially n_windows is never incremented (or decremented) so the 64
map limit is never imposed. Nice idea, shame about the implementation :p

There are also a couple other issues I've spotted in mmap-cache.c so
I'll do some tests and prep a patch shortly.

Col


-- 

Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
  Tribalogic Limited http://www.tribalogic.net/
Open Source:
  Mageia Contributor http://www.mageia.org/
  PulseAudio Hacker http://www.pulseaudio.org/
  Trac Hacker http://trac.edgewall.org/