[Mesa-users] Massive Xorg memory leak pointing to iris_dri

[Radu Rendec]

Hello,

I'm seeing a massive memory leak in the Xorg process. This is not
something new, it's been going on for at least two years, but I didn't
have the time to look closer into it until recently.

I'm using mesa 21.3.1 on Fedora 35. The Xorg process leaks memory,
approx 1 MB every few minutes. On a system with large uptime, the Xorg
process memory usage grows easily by 1 GB within a few days, eventually
filling up most of the available memory.

I believe this is *not* another instance of an X resource leak, for the
following reasons:
 * The leak occurs only when running bare metal on my laptop. I'm
   running the same environment (same Fedora version, same set of
   applications) in a virtualbox VM and in a qemu-kvm VM, and the leak
   does not occur.
 * The `xrestop` program shows no increase of allocated resources, even
   though the Xorg process leaks memory at the same time.

I used a simple malloc/calloc/realloc wrapper that I wrote myself, and
found two spots where a large amount of memory is leaked. Statistics
below are based on data that was gathered within just a few minutes of
running Xorg.

691136 bytes in 5400 occurrences of:
calloc + 130 in section .text of /usr/local/lib64/libdebugalloc.so
iris_create_surface + 795 in section .text of /usr/lib64/dri/iris_dri.so
tc_create_surface + 24 in section .text of /usr/lib64/dri/iris_dri.so
st_update_renderbuffer_surface + 528 in section .text of /usr/lib64/dri/iris_dri.so
st_render_texture + 153 in section .text of /usr/lib64/dri/iris_dri.so
_mesa_framebuffer_texture + 320 in section .text of /usr/lib64/dri/iris_dri.so
_mesa_FramebufferTexture2D + 46 in section .text of /usr/lib64/dri/iris_dri.so
glamor_pixmap_ensure_fb.lto_priv + 136 in section .text of /usr/lib64/xorg/modules/libglamoregl.so
glamor_create_fbo + 123 in section .text of /usr/lib64/xorg/modules/libglamoregl.so
glamor_create_pixmap + 1128 in section .text of /usr/lib64/xorg/modules/libglamoregl.so
ProcCreatePixmap + 254 in section .text of /usr/libexec/Xorg.real
main + 3095 in section .text of /usr/libexec/Xorg.real
__libc_start_call_main + 128 in section .text of /lib64/libc.so.6
__libc_start_main_impl + 124 in section .text of /lib64/libc.so.6
_start + 37 in section .text of /usr/libexec/Xorg.real

198560 bytes in 6205 occurrences of:
realloc + 213 in section .text of /usr/local/lib64/libdebugalloc.so
_iris_batch_flush + 1642 in section .text of /usr/lib64/dri/iris_dri.so
iris_fence_flush + 153 in section .text of /usr/lib64/dri/iris_dri.so
st_glFlush + 55 in section .text of /usr/lib64/dri/iris_dri.so
_glamor_block_handler + 100 in section .text of /usr/lib64/xorg/modules/libglamoregl.so
msBlockHandler + 55 in section .text of /usr/lib64/xorg/modules/drivers/modesetting_drv.so
BlockHandler + 164 in section .text of /usr/libexec/Xorg.real
WaitForSomething + 338 in section .text of /usr/libexec/Xorg.real
main + 1277 in section .text of /usr/libexec/Xorg.real
__libc_start_call_main + 128 in section .text of /lib64/libc.so.6
__libc_start_main_impl + 124 in section .text of /lib64/libc.so.6
_start + 37 in section .text of /usr/libexec/Xorg.real

Both stack traces point to iris_dri.so. However, this may be a leak in
libglamoregl.so, since that's what calls into iris_dri.so in both
cases. That would make it an Xorg bug instead.

For what is worth, I checked the Xorg log, and iris_dri.so is *not*
used in either of the VM environments I mentioned before (where the
leak does not occur). In fact, no DRI is used at all in those
environments.

I am not familiar with the mesa (or Xorg) code base, however I am a
fairly experienced C programmer, so I can help debug this further and
narrow down the problem. I just need some guidance, so any suggestion
would be appreciated.

Thank you,
Radu Rendec