[Bug 728502] New: ximagesrc has a serious shmem leak

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=728502
  GStreamer | gst-plugins-good | 0.10.0

           Summary: ximagesrc has a serious shmem leak
    Classification: Platform
           Product: GStreamer
           Version: 0.10.0
        OS/Version: All
            Status: UNCONFIRMED
          Severity: major
          Priority: Normal
         Component: gst-plugins-good
        AssignedTo: gstreamer-bugs at lists.freedesktop.org
        ReportedBy: alex at stunpix.com
         QAContact: gstreamer-bugs at lists.freedesktop.org
     GNOME version: ---


Created an attachment (id=274672)
 View: https://bugzilla.gnome.org/attachment.cgi?id=274672
 Review: https://bugzilla.gnome.org/review?bug=728502&attachment=274672

ximagesrc shmem leak patch

Seems I found quite old ximagesrc leak that, I suppose, affects all platforms
since 0.10 (or maybe earlier). Here is description of how I discovered it and
some technical background about the bug.

For my arm board with Ubuntu 12.04 I wrote an app that captures a desktop,
encodes video to h264 and streams it to server. Everything went good until I
noticed that app always got killed by system within one or two hours since it
was started. I began to suspect oom_killer. After hours of memory monitoring I
detected that both resident and virtual memory are constantly growing. Most
growing was a virtual memory. By disabling all elements in pipeline I found a
minimal pipeline that leaks:

gst-launch ximagesrc ! queue ! fakesrc

Depending on amount of memory in your system you'll get 'Killed' message after
running this simple pipeline.

I tested this pipeline on following platforms:
 * ubuntu 12.04 arm + gstreamer0.10
 * fresh archlinux x86_64 + gstreamer1.0
 * ubuntu 13.10 x86_64 + gstreamer0.10/1.0

All these distros were affected. One note about ubuntu 13.10 x86_64: it leaks
with much slower rates than other tested distros, but also leaks. I don't know
why, but this is a fact. As I said, I suppose that all other platforms are also
affected, since this bug isn't platform specific.

How to measure. You need to run something that changes the desktop constantly
(looped animated gif for instance). After that you need to run pipeline and
then run oom_score checker:

    cat /proc/$(pidof gst-launch-0.10)/oom_score

you'll notice that score is growing: from 3-5 to 15-20 within few minutes. On
my arm system with 2Gb ram it could take up to 1 hour to reach score ~500 when
process is killed by system.

Next measurement: what is leaking. If we'll run a command:

    cat /proc/$(pidof gst-launch-0.10)/maps | grep SYSV0

we will see something like this:

41d00000-42084000 rw-s 00000000 00:04 316244015  /SYSV00000000 (deleted)
42114000-42498000 rw-s 00000000 00:04 316276788  /SYSV00000000 (deleted)
43000000-43384000 rw-s 00000000 00:04 316309558  /SYSV00000000 (deleted)
4340b000-4378f000 rw-s 00000000 00:04 316342327  /SYSV00000000 (deleted)
4378f000-43b13000 rw-s 00000000 00:04 316375096  /SYSV00000000 (deleted)
43b3e000-43ec2000 rw-s 00000000 00:04 316407866  /SYSV00000000 (deleted)
43f22000-442a6000 rw-s 00000000 00:04 316440636  /SYSV00000000 (deleted)
442f7000-4467b000 rw-s 00000000 00:04 316506174  /SYSV00000000 (deleted)
44738000-44abc000 rw-s 00000000 00:04 316538943  /SYSV00000000 (deleted)

These records are shared memory blocks occupied by the process. After several
minutes you'll see many of them. They aren't freed and counted by system as
used. In normal conditions there should be 2-3 such blocks.

Why this happens. An ximagesrc element uses XShm extension (if available in
system) to get desktop shots. Each such image (gstbuffer actually) has a memory
shared with X server. Once pipeline frees buffer, it returned to ximagesrc and
pushed into pool of unused buffers. Each time when element is asked for a new
frame, it takes from pool first unused buffer and deletes all other buffers.
Here is piece of code responsible for that:

  while (ximagesrc->buffer_pool != NULL) {
    ximage = ximagesrc->buffer_pool->data;

    if ((ximage->width != ximagesrc->width) ||
        (ximage->height != ximagesrc->height)) {
      gst_ximage_buffer_free (ximage);
    }

    ximagesrc->buffer_pool = g_slist_delete_link (ximagesrc->buffer_pool,
        ximagesrc->buffer_pool);
  }

There only one issue: g_slist_delete_link() only removes item from linked list,
frees memory used by slist record, but it doesn't unrefs the buffer which it
holds. Buffer lost, shmem not freed as well. Not detectable by valgrind.

Patch with fix for 0.10.31 attached.

Any comments are appreciated.

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.