[Mesa-dev] [PATCH] winsys/sw/xlib: Prevent shared memory segment leakage.

[jfonseca at vmware.com]

From: José Fonseca <jfonseca at vmware.com>

Running piglit with was causing all sort of weird stuff happening to my
desktop (Chromium webpages become blank, Qt Creator flickered, etc).  I
tracked this down to shared memory segment leakage when GL is not shutdown
properly. The segments can be seen running `ipcs` and looking for
nattch==0.

This changes fixes this by calling shmctl(IPC_RMID) soon after creation
(which does not remove the segment immediately, but simply marks it for
removal when no more processes are attached).

src/mesa/drivers/x11/xm_buffer.c also does something similar, albeit
slightly later in time -- after XShmCreateImage() call --, but calling
shmctl(IPC_RMID) before XShmCreateImage() seems to work equally well,
with fewer chances of leaking the shared memory segment.

NOTE: This is a candidate for stable branches.
---
 src/gallium/winsys/sw/xlib/xlib_sw_winsys.c |   11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/src/gallium/winsys/sw/xlib/xlib_sw_winsys.c b/src/gallium/winsys/sw/xlib/xlib_sw_winsys.c
index 3aef8da..584fbee 100644
--- a/src/gallium/winsys/sw/xlib/xlib_sw_winsys.c
+++ b/src/gallium/winsys/sw/xlib/xlib_sw_winsys.c
@@ -131,8 +131,13 @@ alloc_shm(struct xlib_displaytarget *buf, unsigned size)
    }
 
    shminfo->shmaddr = (char *) shmat(shminfo->shmid, 0, 0);
+
+   /* Mark the segment to be destroyed, so that it is automatically destroyed
+    * when this process dies.
+    */
+   shmctl(shminfo->shmid, IPC_RMID, 0);
+
    if (shminfo->shmaddr == (char *) -1) {
-      shmctl(shminfo->shmid, IPC_RMID, 0);
       return NULL;
    }
 
@@ -165,6 +170,8 @@ alloc_shm_ximage(struct xlib_displaytarget *xlib_dt,
                                       &xlib_dt->shminfo,
                                       width, height);
    if (xlib_dt->tempImage == NULL) {
+
+	   debug_printf("XShmCreateImage failed\n");
       xlib_dt->shm = False;
       return;
    }
@@ -177,6 +184,7 @@ alloc_shm_ximage(struct xlib_displaytarget *xlib_dt,
    XSync(xlib_dt->display, False);
 
    if (XErrorFlag) {
+	   debug_printf("XErrorFlag set\n");
       /* we are on a remote display, this error is normal, don't print it */
       XFlush(xlib_dt->display);
       XErrorFlag = 0;
@@ -251,7 +259,6 @@ xlib_displaytarget_destroy(struct sw_winsys *ws,
    if (xlib_dt->data) {
       if (xlib_dt->shminfo.shmid >= 0) {
          shmdt(xlib_dt->shminfo.shmaddr);
-         shmctl(xlib_dt->shminfo.shmid, IPC_RMID, 0);
          
          xlib_dt->shminfo.shmid = -1;
          xlib_dt->shminfo.shmaddr = (char *) -1;
-- 
1.7.10.4