[Mesa-dev] [PATCH 00/14] radeon/winsyses: sub-allocation for small buffers

[Nicolai Hähnle]

Hi,

there are a bunch of games that allocate huge numbers of small buffers. This
is especially problematic on amdgpu, where we tend to keep mmaps of buffers
around for performance, and we hit the limit of 64k simultaneously open
mmaps. More generally, it leads to very long buffer lists on submissions and
the associated work in the kernel.

This series introduces sub-allocation for buffers at the winsys level. That
is, the winsys presents a regular pb_buffer to the driver, but internally
allocates it from a bigger "real" buffer.

Fences are managed for the small, sub-allocated buffers: we don't want to
have to wait for unrelated buffers that happen to be in the same slab as
another sub-allocated buffer that is still busy.

Fences are also used for the parent buffers, but only to make sure that they
are only really freed/reclaimed once they are fully idle.

Sub-buffers do not go into the pb_cache when freed. This can be argued either
way, but I wanted to keep sequential allocations in the same slab where
possible (for locality), and this is easier with a separate reclaim list.
When all entries in a large buffer are freed, the large buffer does go into
the pb_cache.

There is one aspect where the pb_buffer abstraction becomes leaky: getting a
handle to a sub-allocated buffer is problematic in terms of process isolation.
Right now, only textures are shared across process boundaries, and so
textures are never placed into sub-allocated buffers. In the longer run, if
we want to export buffers across process boundaries, we should probably
revisit that and e.g. copy (API level) buffers into their own (kernel leve)
buffer object when necessary. For now, the winsys just adds an offset.

Please review! Thanks,
Nicolai
--
 src/gallium/auxiliary/Makefile.sources       |   2 +
 src/gallium/auxiliary/pipebuffer/pb_cache.h  |   2 +-
 src/gallium/auxiliary/pipebuffer/pb_slab.c   | 195 ++++++++++
 src/gallium/auxiliary/pipebuffer/pb_slab.h   | 115 ++++++
 src/gallium/auxiliary/util/u_math.h          |  12 +
 src/gallium/drivers/r300/r300_emit.c         |  19 +-
 src/gallium/drivers/r300/r300_texture.c      |   2 +-
 src/gallium/drivers/radeon/r600_cs.h         |   6 +-
 src/gallium/drivers/radeon/r600_texture.c    |   2 +
 src/gallium/drivers/radeon/radeon_uvd.c      |   3 +-
 src/gallium/drivers/radeon/radeon_vce.c      |   3 +-
 src/gallium/drivers/radeon/radeon_winsys.h   |  11 +-
 src/gallium/winsys/amdgpu/drm/amdgpu_bo.c    | 330 +++++++++++++---
 src/gallium/winsys/amdgpu/drm/amdgpu_bo.h    |  47 ++-
 src/gallium/winsys/amdgpu/drm/amdgpu_cs.c    | 248 +++++++++---
 src/gallium/winsys/amdgpu/drm/amdgpu_cs.h    |  26 +-
 .../winsys/amdgpu/drm/amdgpu_winsys.c        |  13 +
 .../winsys/amdgpu/drm/amdgpu_winsys.h        |   5 +
 .../winsys/radeon/drm/radeon_drm_bo.c        | 331 +++++++++++++++--
 .../winsys/radeon/drm/radeon_drm_bo.h        |  40 +-
 .../winsys/radeon/drm/radeon_drm_cs.c        | 228 +++++++++---
 .../winsys/radeon/drm/radeon_drm_cs.h        |  16 +-
 .../winsys/radeon/drm/radeon_drm_winsys.c    |  26 +-
 .../winsys/radeon/drm/radeon_drm_winsys.h    |   7 +
 24 files changed, 1449 insertions(+), 240 deletions(-)