[pulseaudio-discuss] [PATCH 00/11] Introduce memfd support

[Ahmed S. Darwish]

Hi everyone,

This RFC patch series introduces memfd support [*] to PulseAudio,
laying out the necessary (but not yet sufficient) groundwork for
sandboxing, protecting PulseAudio from its clients, and protecting
clients (data) from each other.

Memfd support is added in quite a transparent manner, respecting
current PA mechanisms and abstractions. The lower-level layers are
properly refactored and extended: the srbchannel communication path
is transformed to memfds by only changing a single line of code.

Patches 1 to 11 were tested using parallel execution of 2 gstreamer
clients (gnome-music, gst-launch), 2 mplayer clients with `-ao pulse',
and 2 pulse `paplay' clients. Backward compatibility with earlier
library versions is fully maintained, including srbcahannel posix
shm compatibility, and was tested using an Arch Linux userland.

These patches touch a lot of the core PA code paths, so please give
them your much needed reviews and scrutiny :-)

##

In general, this series can be divided to:

- Patches #1-#2 reduces shared memory between clients by transforming
  the server-wide srbchannel mempool to a per-client one.

- Patches #3-#6 refactors pa_mempool to use three memory backends:
  pa_shm, pa_memfd, and pa_privatemem. pa_shm code is also refactored
  to be responsbile only for POSIX shm_open(2) memory.

- Patch #7 introduces the actual memfd support!

- Patch #8-#10 introduces memfd-backed memblocks support to memexports,
  memimports, and pstreams.

- Patch #11 enables memfd for the srbchannel code and pumps protocol
  version while maintaining comapatibility.

- Debugging patch #12 enables memfd for client's audio data. Please
  check the comments on this 'patch' for further details, some help is
  quite needed there :-)

##

TODO in further iterations:

- Split the daemon's core mempool into a per-client mempool and a
  private one. This way, zero memory regions are shared between clients
  and recording access can be further protected.

- Latency tests. As discussed with Arun in latest weekly meeting, this
  can be done either through manual methods, code tracing, perf (yay!),
  or within the the testing framework dummy daemon

- Memory benchmarks; actual test-cases under `src/tests', and a bugfix
  for debugging patch #12

- Memfd sealing .. without changing the current protocol semantics

- Long-term: slight protocol changes to pass fds only from the server
  to the client, and not in the other direction like what is now done
  for playback buffers. Check the discussion at the bottom of
  https://plus.google.com/+DavidHerrmann/posts/h8CXYqwXruw for context.

##

Global diffstat:

Ahmed S. Darwish (11):
  pulsecore: Cache daemon shm size inside pa_core
  pulsecore: srbchannel: Introduce per-client SHM files
  pulsecore: Transform pa_mempool_new() into a factory method
  pulsecore: Split pa_shm mempool backend into pa_shm and pa_privatemem
  pulsecore: Provide an abstract interface for pa_shm and pa_privatemem
  pulsecore: Extract generic bits from pa_shm into its parent pa_mem
  pulsecore: Introduce memfd support
  pulsecore: Standardize memory-types annotation using pa_mem_type_t
  pulsecore: memexport/memimport: Introduce memfd blocks support
  pulsecore: pstreams: Introduce memfd blocks support
  pulsecore: srbchannel: Enable memfd support; pump protocol version

 PROTOCOL                        |  15 ++
 configure.ac                    |  21 ++-
 src/Makefile.am                 |   8 ++
 src/pulse/context.c             |  12 +-
 src/pulsecore/client.c          |   5 +
 src/pulsecore/client.h          |   7 +
 src/pulsecore/core.c            |  20 ++-
 src/pulsecore/core.h            |  11 +-
 src/pulsecore/mem.c             | 108 ++++++++++++++
 src/pulsecore/mem.h             |  64 +++++++++
 src/pulsecore/memblock.c        | 307 +++++++++++++++++++++++++++++++++-------
 src/pulsecore/memblock.h        |  12 +-
 src/pulsecore/memfd-wrappers.h  |  72 ++++++++++
 src/pulsecore/memfd.c           | 108 ++++++++++++++
 src/pulsecore/memfd.h           |  68 +++++++++
 src/pulsecore/privatemem.c      |  82 +++++++++++
 src/pulsecore/privatemem.h      |  35 +++++
 src/pulsecore/protocol-native.c |  45 +++++-
 src/pulsecore/pstream.c         | 195 +++++++++++++++++++------
 src/pulsecore/shm.c             | 180 ++++++-----------------
 src/pulsecore/shm.h             |  14 +-
 src/tests/cpu-mix-test.c        |   2 +-
 src/tests/lfe-filter-test.c     |   2 +-
 src/tests/mcalign-test.c        |   2 +-
 src/tests/memblock-test.c       |  14 +-
 src/tests/memblockq-test.c      |   2 +-
 src/tests/mix-test.c            |   2 +-
 src/tests/remix-test.c          |   2 +-
 src/tests/resampler-test.c      |   2 +-
 src/tests/srbchannel-test.c     |   2 +-
 30 files changed, 1134 insertions(+), 285 deletions(-)
 create mode 100644 src/pulsecore/mem.c
 create mode 100644 src/pulsecore/mem.h
 create mode 100644 src/pulsecore/memfd-wrappers.h
 create mode 100644 src/pulsecore/memfd.c
 create mode 100644 src/pulsecore/memfd.h
 create mode 100644 src/pulsecore/privatemem.c
 create mode 100644 src/pulsecore/privatemem.h

[*] memfd_create(2), David Herrmann blog
    https://dvdhrm.wordpress.com/2014/06/10/memfd_create2/

Regards,
--
Ahmed Darwish
http://darwish.chasingpointers.com