[pulseaudio-tickets] [PulseAudio] #337: /usr/lib/libpulse.so.0 causes applications to leak memory

[PulseAudio]

#337: /usr/lib/libpulse.so.0 causes applications to leak memory
-------------------------------------+--------------------------------------
 Reporter:  anstedt                  |       Owner:  lennart
     Type:  defect                   |      Status:  new    
 Priority:  normal                   |   Milestone:         
Component:  libpulse                 |    Severity:  normal 
 Keywords:  memory,leak,application  |  
-------------------------------------+--------------------------------------
 First lets prove the Pulse Audio library leaks. (/usr/lib/libpulse.so.0)

 1. Configure ALSA to use Pulse Audio
 2. Look at the code in pcm_min.c, notice it does nothing more than open
 and
    configure the audio interface then goes into a loop for 120 minutes.
 2. Compile pcm_min: gcc -lasound pcm_min.c -o pcm_min
 3. execute ./pcm_min in one shell
 4. In a second shell execute the following:
    while [ 1 ]; do ps axO-R -o vsize,args | grep pcm_min | grep -v grep;
 sleep 10; done
    This outputs VSZ every 10 seconds. VSZ is the amount of allocated
 memory use
    by the application. In this case it should not change after startup.
 5. Notice the VSZ continues to increase over time. This is the memory
 leak.

 Where is the leak?

 First let me say that I am no expert on Pulse Audio. But here is what I
 understand.

 The library libpulse.so.0 spawns off at least one thread and creates some
 sort of
 timed event queue. The queue it uses is an idxset container supplied by
 the
 libpulse.so.0 library. It adds some sort of repeating time event into the
 queue,
 this event is never removed since it creates a needed 10 Hz time event.
 Other
 single shot time events are then added to and removed from this same
 queue. The
 queue never contains more than 10 or 20 entries.

 Why does this cause a leak?

 idxset is a multi-purpose container. One of its functions is to act like a
 variable length array which can be accessed via index's. It never reuses
 an index
 and does handle new entries being added as long as ALL the old entries are
 added.

 Here is an example of normal idxset usage:
 The maximum memory uses will be for the 200 entries in the idxset
 container
 called queue.

 main()
 {
   int i = 0;
   int j = 0;
   int my_data[200];
   uint32_t *idx;

   pa_idxset* queue = pa_idxset_new(0, 0);

   /* Add and remove 200 entries 100 times */
   for (j=0; j <=100; j++)
   {
     for (i=0; i<200; i++)
     {
       pa_idxset_put(queue, &(my_data[i]), 0);
     }

     for (i=0; i<200; i++)
     {
       pa_idxset_get_by_data(queue, &(my_data[i]), &idx)
       pa_idxset_remove_by_index(queue, idx);
     }
    }

    /* at this point the total memory used will be for 200 entries */

    pa_idxset_free(queue, 0, 0);
 }

 Lets make it leak:

 main()
 {
   int i = 0;
   int j = 0;
   int my_data[200];
   uint32_t *idx;

   pa_idxset* queue = pa_idxset_new(0, 0);

   /* add the first entry and never remove it */
   pa_idxset_put(queue, &(my_data[0]), 0);

   /* Add and remove 200 entries 100 times */
   for (j=0; j <=100; j++)
   {
     for (i=1; i<200; i++)
     {
       pa_idxset_put(queue, &(my_data[i]), 0);
     }

     for (i=1; i<200; i++)
     {
       pa_idxset_get_by_data(queue, &(my_data[i]), &idx)
       pa_idxset_remove_by_index(queue, idx);
     }
    }

    /* At this point the total memory used will be close to (100 * 200)
 entries.
       If you increase the outer loop the memory used will also increase */

    /* Now remove it */
    pa_idxset_get_by_data(queue, &(my_data[0]), &idx)
    pa_idxset_remove_by_index(queue, idx);

    pa_idxset_free(queue, 0, 0);
 }

 Why does this leak occur we still have the same number of entries? idxset
 always
 hands out unique indexes for every call to pa_idxset_put(). Then why
 doesn't the
 first case leak since you still handed out a large number of indexes? But
 idxset
 will use the lowest index it has as the very first entry in its internal
 buffer.
 In the first case you add in 200 entries and get back index's 0...199. The
 buffer needs to be 200 deep since you have used indexes 0...199. Then when
 you
 delete them all and add in a second set of 200 entries you now have
 indexes
 200...399. The internal buffer uses index 200 for the first entry into the
 first
 location in the internal buffer ans so on for each of the 200 entries.
 idxset
 still only needs an internal buffer of 200.

 In the second case we insert 0 then 1...199 and idxset needs an internal
 buffer
 of 200 now we delete all entries/indexes other than index 0 and put back
 1...199
 we then need new indexes 200...398 not 399 since we only added back in 199
 entries. idxset then needs an internal buffer of 398 since it needs to
 cover
 indexes 0...398. It uses indexes as true indexes and not keys so it needs
 to
 expand the internal buffer. So every time you remove all entries other
 than 0
 and add them back in again the idxset buffer needs to expand for all the
 indexes.

 This is exactly what is done by the timed event mechanism, actually I
 think it
 leaves entry/index 4 in the buffer forever but the effect is the same.

 I have included a modified idxset which rather than using the indexes as
 true
 indexes it uses them as keys and reuses the keys after they are no longer
 needed. In most cases idxset will be slightly faster but there are case
 where
 the buffer needs to be scanned to find and empty slot which can be slower.

 Even thought these changes are for Pulse Audio version 9.9.2, idxset is
 basically the same. I have to admit I do not know if the timed event
 scheme is
 any different in later versions and would avoid this problem.

 You can email me at howard.anstdet at med.ge.com if you want help with this
 problem.
 It will show up in any system which runs an audio application for a long
 time
 since it leaks 400 bytes every 10 seconds.

 Our application needs to run many months and as such runs out of memory
 without
 this change.

 Sorry for the long text but this is an extremely subtle problem.

-- 
Ticket URL: <http://www.pulseaudio.org/ticket/337>
PulseAudio <http://pulseaudio.org/>
The PulseAudio Sound Server