[pulseaudio-discuss] [PATCH v3 00/11] Introduce memfd support

[Ahmed S. Darwish]

Hello!

The simplified memfd patch series ;-)

==> v3 changes:

- Further testing

  Testing methods are summarized in below Google Sheet:

              https://goo.gl/Ls6gzJ

- Fault injection tests showed that PA design does not guarantee
  per-client mempool's blocks being referenced only by client
  specific objects. Thus pools are now reference-counted.

  (patch #2)
  Ref: http://article.gmane.org/gmane.comp.audio.pulseaudio.general/25270

- Vacuuming of, the now per-client, srbchannel mempool is removed.
  This will only be needed when transforming the global mempool to
  per-client model, so let's do it there

  (patch #3)
  Ref: http://article.gmane.org/gmane.comp.audio.pulseaudio.general/25153

- SHM and memblock.c code further simplified: pa_shm and pa_mempool
  are now void of any `per-type' constructs or `privatemem' object
  abstractions, etc.

  (patch #4, patch #5)
  Ref: http://article.gmane.org/gmane.comp.audio.pulseaudio.general/25155

- For memfd support, error recovery paths has been clarified.
  Concise comments regarding fd ownership, in different areas of the
  code, were also added

  (patch #6)
  Ref: http://article.gmane.org/gmane.comp.audio.pulseaudio.general/25253

- For on demand SHM ID to memfd mappings, no more special packets
  are created or used. A new command, PA_COMMAND_REGISTER_MEMFD_SHMID,
  is used instead

  (patch #9)
  Ref: http://article.gmane.org/gmane.comp.audio.pulseaudio.general/25181

- To accomplish the above without any fd leaks, fds are now included
  in the PA ancillary structure itself. A new fd cleanup strategy for
  such an object is then introduced.

  pa_cmsg_ancil_data_close_fds() is introduced and called in multiple
  error recovery and cleanup paths while being multiple-invocations
  safe.

  (patch #9)
  Ref: http://article.gmane.org/gmane.comp.audio.pulseaudio.general/25274

- 'enable-memfd=' is now used for daemon and client configurations
  instead of 'disable-memfd=', thus avoiding confusing double
  negative grammar

  (patch #10, patch #11)
  Ref: http://article.gmane.org/gmane.comp.audio.pulseaudio.general/25165

- v2 BugFix: when setting 'enable-memfd=no' in the client, v2 still
  erroneously informed the server of supporting memfd transport.
  This is now fixed.


==> references:

- v2 submission
  http://thread.gmane.org/gmane.comp.audio.pulseaudio.general/25131

- v1 submission
  http://thread.gmane.org/gmane.comp.audio.pulseaudio.general/24110


==> diffstat:

Ahmed S. Darwish (11):
  pulsecore: Cache daemon shm size inside pa_core
  pulsecore: Reference count mempools
  srbchannel: Introduce per-client SHM files
  pulsecore: Transform pa_mempool_new() into a factory method
  SHM: Refactor private allocations
  pulsecore: Introduce memfd support
  memimport: Support memfd blocks
  pulsecore: Specially mark global mempools
  pstream: Support memfd blocks transport
  client audio: Use memfd transport by default; pump protocol version
  core: Use memfd transport by default

 PROTOCOL                          |  16 +++
 configure.ac                      |  21 ++-
 man/pulse-client.conf.5.xml.in    |   8 ++
 man/pulse-daemon.conf.5.xml.in    |   7 +-
 man/pulseaudio.1.xml.in           |  15 +-
 shell-completion/bash/pulseaudio  |   4 +-
 shell-completion/zsh/_pulseaudio  |   1 +
 src/Makefile.am                   |   8 +-
 src/daemon/cmdline.c              |  13 +-
 src/daemon/daemon-conf.c          |   1 +
 src/daemon/daemon-conf.h          |   1 +
 src/daemon/main.c                 |   4 +-
 src/modules/module-tunnel.c       |   2 +-
 src/pulse/client-conf.c           |   1 +
 src/pulse/client-conf.h           |   2 +-
 src/pulse/context.c               | 103 +++++++++++---
 src/pulse/internal.h              |   3 +
 src/pulsecore/core.c              |  24 ++--
 src/pulsecore/core.h              |  13 +-
 src/pulsecore/creds.h             |   5 +
 src/pulsecore/filter/lfe-filter.c |   5 +-
 src/pulsecore/iochannel.c         |   1 +
 src/pulsecore/mem.h               |  60 ++++++++
 src/pulsecore/memblock.c          | 277 ++++++++++++++++++++++++++++++++---
 src/pulsecore/memblock.h          |  22 ++-
 src/pulsecore/memblockq.c         |   5 +-
 src/pulsecore/memchunk.c          |   5 +-
 src/pulsecore/memfd-wrappers.h    |  68 +++++++++
 src/pulsecore/native-common.c     |  78 ++++++++++
 src/pulsecore/native-common.h     |  11 ++
 src/pulsecore/pdispatch.c         |  30 ++--
 src/pulsecore/pdispatch.h         |   5 +-
 src/pulsecore/protocol-native.c   | 109 ++++++++++++--
 src/pulsecore/pstream-util.c      | 100 ++++++++++++-
 src/pulsecore/pstream-util.h      |   4 +-
 src/pulsecore/pstream.c           | 200 ++++++++++++++++++++++----
 src/pulsecore/pstream.h           |   8 +-
 src/pulsecore/shm.c               | 293 +++++++++++++++++++++++---------------
 src/pulsecore/shm.h               |  23 ++-
 src/tests/connect-stress.c        |   9 +-
 src/tests/cpu-mix-test.c          |   4 +-
 src/tests/lfe-filter-test.c       |   4 +-
 src/tests/mcalign-test.c          |   4 +-
 src/tests/memblock-test.c         |  21 +--
 src/tests/memblockq-test.c        |   4 +-
 src/tests/mix-test.c              |   4 +-
 src/tests/remix-test.c            |   4 +-
 src/tests/resampler-test.c        |   4 +-
 src/tests/srbchannel-test.c       |   6 +-
 49 files changed, 1335 insertions(+), 285 deletions(-)
 create mode 100644 src/pulsecore/mem.h
 create mode 100644 src/pulsecore/memfd-wrappers.h
 create mode 100644 src/pulsecore/native-common.c

Regards,

--
http://darwish.chasingpointers.com