[systemd-devel] [SPECIFICATION RFC v3] The firmware and bootloader log specification

[Alec Brown]

Hi everyone,

I've been working on improving the specification for the firmware and bootloader
log that Daniel Kiper has proposed and take into account most of the suggestions
that were made in these threads [1], [2].

The goal is to allow various boot components to pass logs to the running OS and
then to the user space for processing and analysis. These logs should be generic
enough so that it can work in multiple environments and be human readable.

It has yet to be decided where to put the final version of this specification.
It should be merged into an existing specification, e.g. UEFI, ACPI, Multiboot2,
or be standalone, such as a part of OASIS Standards.

Below is how the layout of these logs would store their data.

bf_log_header:
               +-------------------+
u32            | version           |
u32            | size              |
u8[64]         | producer          |
u8[64]         | log_format        |
u64            | flags             |
u64            | next_bflh_addr    |
u64            | log_addr          |
u32            | log_size          |
               +-------------------+

bf_log_buffer:
               +-------------------+
u32            | version           |
u32            | size              |
u8[64]         | producer          |
u32            | next_msg_off      |
bf_log_msg[l]  | msgs              |
               +-------------------+

bf_log_msg:
               +-------------------+
u32            | size              |
u64            | ts_nsec           |
u32            | level             |
u32            | facility          |
u32            | msg_off           |
u8[n]          | type              |
u8[m]          | msg               |
               +-------------------+

Where l is the number of msgs, n is the size of type, and m is the size of the
msg.

The bf_log_header structure forms a linked list. Each bf_log_header element in
the linked list points to the individual log buffer and the next bf_log_header
element in the linked list. The first element in the linked list points to the
last boot component in the boot chain. The last element points to the starting
boot component in the boot chain. The log buffers which contain the log
messages are produced by the various boot components, typically from the
firmware to the bootloader. The log message is stored in a log format that is
compatible with the boot component that produced it.

The fields in bf_log_header structure:
  - version: the firmware and bootloader log header version number, 1 for now,
  - size: the size of the bf_log_header to allow for backward compatibility if 
    other fields are added,
  - producer: the producer/firmware/bootloader/... entity, NUL terminated
    string, e.g. GRUB, Coreboot; the length allows for ASCII UUID storage,
  - log_format: the format used to record the log messages, NUL terminated
    string, e.g. bf_log_msg, cbmem_cons, etc.; various producers may generate
    logs in various formats if needed,
  - flags: bit field used to store information about the log state, if bit 0 has
    been set it means the log was truncated,
  - next_bflh_addr: the physical address of the next bf_log_header structure,
    none if zero,
  - log_addr: the physical address of where the log buffer is stored,
  - log_size: the total size of the log buffer.

The bf_log_buffer is used to store log messages from the firmware and
bootloader. This format for storing messages is called the bf log format. The
bf_log_buffer contains the header information of the bf log format with the log
messages being stored in an array of bf_log_msg messages.

The fields in bf_log_buffer structure:
  - version: the firmware and bootloader log version number, 1 for now,
  - size: the total allocated space for the bf_log_buffer including the log
    messages stored in msgs,
  - producer: the producer/firmware/bootloader/... entity, NUL terminated
    string, e.g. GRUB, Coreboot; the length allows for ASCII UUID storage; same
    as the field in bf_log_header,
  - next_msg_off: the byte offset from the beginning of the allocated space for
    bf_log_buffer to the next byte after the last bf_log_msg in msgs,
  - msgs: the array of log messages stored in the bf_log_msg structures.

The fields in bf_log_msg structure:
  - size: the total size of the bf_log_msg entry,
  - ts_nsec: the timestamp in nanoseconds starting from 0 (zero); the producer
    using this log format defines the meaning of 0,
  - level: similar to the syslog meaning; used to differentiate normal log
    messages from debug log messages, but the exact interpretation depends on
    the producer,
  - facility: similar to the syslog meaning; used to differentiate the sources
    of the log messages, but the exact interpretation depends on the producer,
  - msg_off: the byte offset which the msg field starts in bf_log_msg,
  - type: the log message type; similar to facility but NUL terminated string
    instead of integer, but the exact interpretation depends on the producer,
  - msg: the log message, NUL terminated string.

In bf_log_msg, the producers are free to use or ignore any of the level,
facility, and type fields. If level or facility are ignored, they should be set
to 0. If type is ignored, it should be set to an empty NUL terminated string.

Since it doesn't seem possible to have each boot component using the same log
format, we added a log_format and log_phys_addr fields to give flexibility in
how logs are stored. An example of a different log format that can be used is
the cbmem_console log format used by coreboot:

cbmem_console:
               +-------------------+
u32            | size              |
u32            | cursor            |
u8[m]          | body              |
               +-------------------+

There is still the outstanding issue of how the logs will be sent to the OS. If
UEFI is used, we can use config tables. If ACPI or Device Tree is used, we can
use bf_log_header.next_bflh_addr to present the logs. If none of these platforms
are used, it becomes a lot trickier to solve this issue.

Any suggestions are much appreciated and will be taken into consideration.

I will be presenting this work at the LPC System Boot and Security
Micro-conference on the 22nd of September at 7:50 AM PDT (14:50 UTC). Come and
join if you want to discuss the design. The schedule for the System Boot and
Security Micro-conference can be found here [3].

Thanks!
Alec Brown

[1] https://lists.gnu.org/archive/html/grub-devel/2020-11/msg00100.html
[2] https://lists.gnu.org/archive/html/grub-devel/2020-12/msg00021.html
[3] https://linuxplumbersconf.org/event/11/sessions/116/#20210922