[systemd-devel] [PATCH v4] journalctl: Add support for showing messages from a previous boot

Fri Jun 14 13:43:32 PDT 2013

On Wed, Jun 12, 2013 at 02:41:15PM +0200, Jan Janssen wrote:
> Unfortunately, to get a chronological list of boot IDs, we
> need to search through the journal. sd_journal_enumerate_unique()
> doesn't help us here, because the order of returned values
> is undefined.
Hi Jan,

as an experiment, I tried to create the time-sorted list of boot
ids in python:

from pprint import pprint
from systemd import journal
j = journal.Reader()
matches={}
for id in j.query_unique('_BOOT_ID'):
   j.add_match(_BOOT_ID=id.hex)
   j.seek_head()
   try:
       ts = j.get_next()['__REALTIME_TIMESTAMP']
       print(ts)
   except OSError as e:
       print(e)    # [1]
   else:
       matches[ts] = id
   j.flush_matches()

pprint(sorted(matches.items()))

This runs very fast on my machine... I have ~1GB of logs (cached in RAM),
and it runs .12s including Python load time. In fact python -c '' takes .04s,
so not an insubstantial chunk of the time for the whole query.

I think you might be overcomplicating the function to query boot
ids.

> An initial search for the reference boot ID is performed. We then
> start a search filtering by SD_MESSAGE_JOURNAL_START. This
> message ID should come up in every journal and is therefore a good
> start to reduce the amount of messages the lookup process has to
> walk through to find the previous/next boot IDs.
> 
> Note that this or any other message ID could get rotated away,
> so lookup is not guaranteed to be precise. This should only affect
> old (and uninteresting) journal entries, though.
Well, I find that part a bit iffy. One man's old and uninteresting
journal entry is another man's memorable boot from last Monday.

(I'm completely ignoring the user interface issues for now.)

Zbyszek

[1] I get:

2013-05-14 20:02:47.550789
2013-06-11 22:04:04.936109
2013-04-22 18:35:15.667285
2013-04-24 18:54:33.018465
2013-04-26 22:20:56.126600
2013-04-23 21:13:41.322469
2013-05-24 02:28:00.288378
2013-05-24 02:07:49.595858
journal gave us a field without '='
journal gave us a field without '='
2013-04-22 18:39:08.894320
2013-04-23 18:21:49.213912
2013-05-14 20:06:08.123713
2013-04-26 22:11:57.116450
2013-04-10 17:57:36.943758
2013-04-26 22:09:12.834640
2013-04-25 18:25:54.288430
2013-05-14 20:09:09.786221
2013-05-24 02:02:37.057646
2013-04-24 18:57:09.434416
2013-04-10 18:11:03.702147
2013-05-14 20:12:10.394994
2013-04-24 19:22:05.785226
2013-04-10 17:39:35.400263
journal gave us a field without '='
2013-04-25 18:21:39.809359
2013-05-29 22:39:41.825855
[(datetime.datetime(2013, 4, 10, 17, 39, 35, 400263), UUID('2944cf29-ee75-416d-9b77-aa68f4cc09f3')),
 (datetime.datetime(2013, 4, 10, 17, 57, 36, 943758), UUID('147b961e-cd0a-4741-8c83-dfd7f7a2e49f')),
 (datetime.datetime(2013, 4, 10, 18, 11, 3, 702147),  UUID('17d93b1d-45a4-439e-b592-a3aad414adde')),
 (datetime.datetime(2013, 4, 22, 18, 35, 15, 667285),  UUID('805564d7-e296-4379-a690-803478ba1020')),
 (datetime.datetime(2013, 4, 22, 18, 39, 8, 894320),  UUID('e192d67b-0429-4a66-a768-bbdbdd3b40b7')),
 (datetime.datetime(2013, 4, 23, 18, 21, 49, 213912),  UUID('b95b07e7-f142-4bba-afc6-741178f763a6')),
 (datetime.datetime(2013, 4, 23, 21, 13, 41, 322469),  UUID('a4690eff-ff59-40ad-a755-bcd029f3b1c3')),
 (datetime.datetime(2013, 4, 24, 18, 54, 33, 18465),  UUID('e9f0a679-0b4e-4082-9d02-2c42802898c0')),
 (datetime.datetime(2013, 4, 24, 18, 57, 9, 434416),  UUID('5e0cc564-9d1a-4aa0-bc6e-9ee916dc013a')),
 (datetime.datetime(2013, 4, 24, 19, 22, 5, 785226),  UUID('45c46331-41e3-4ea6-8e5d-8f385eec7397')),
 (datetime.datetime(2013, 4, 25, 18, 21, 39, 809359),  UUID('3a3e35fe-f06e-4a01-9d96-f76080497bda')),
 (datetime.datetime(2013, 4, 25, 18, 25, 54, 288430),  UUID('3d7210f8-b201-434a-aa27-7b310b0b4ce7')),
 (datetime.datetime(2013, 4, 26, 22, 9, 12, 834640),  UUID('b96f6648-12a5-4062-b15e-983e83dd48f5')),
 (datetime.datetime(2013, 4, 26, 22, 11, 57, 116450),  UUID('9f788e82-05c5-4008-adc2-540c358740c1')),
 (datetime.datetime(2013, 4, 26, 22, 20, 56, 126600),  UUID('5e247c3a-7228-4bdf-9c17-7baa503fc2fd')),
 (datetime.datetime(2013, 5, 14, 20, 2, 47, 550789),  UUID('8f31c0fc-2bf7-4369-b6ea-02c660efc016')),
 (datetime.datetime(2013, 5, 14, 20, 6, 8, 123713),  UUID('d3458ed2-b2ab-4e0e-a889-61c1c81b8780')),
 (datetime.datetime(2013, 5, 14, 20, 9, 9, 786221),  UUID('ce419d19-b729-4ebe-9c5b-09d76de1bc9c')),
 (datetime.datetime(2013, 5, 14, 20, 12, 10, 394994),  UUID('08e35313-2ba8-4bb6-8b6e-5eabd9c01325')),
 (datetime.datetime(2013, 5, 24, 2, 2, 37, 57646),  UUID('34b39911-2aaa-40ed-938d-73ee753519d2')),
 (datetime.datetime(2013, 5, 24, 2, 7, 49, 595858),  UUID('51eb5172-a91e-4eab-b2e6-6d53af26b32d')),
 (datetime.datetime(2013, 5, 24, 2, 28, 0, 288378),  UUID('06de18da-36a5-46d4-9788-cc3aa9d15473')),
 (datetime.datetime(2013, 5, 29, 22, 39, 41, 825855),  UUID('4e3c518a-b047-4c12-ac8d-e7896fe6b154')),
 (datetime.datetime(2013, 6, 11, 22, 4, 4, 936109),  UUID('44eedc13-74cf-4a32-954c-9f299a228565'))]

... which means that there's some corruption in my journals.
Not really surprising, I think it might be sizeof() bug of yore.
The field returned looks like '_MACHINE_ID5833158886a8445e801d437313d25eff'.
Ignoring that for now.