[pulseaudio-discuss] Hard-coded allocation of 64 MB of memory upfront too big in some scenarios

[Jim Carter]

On Wed, 27 Aug 2008, Lennart Poettering wrote:
> On Wed, 27.08.08 10:15, rdiezmail-pulseaudio at yahoo.de (rdiezmail-pulseaudio at yahoo.de) wrote:
> > As I stated on my original message, my embedded PowerPC target
> > hasn't got shared memory (i.e. not configured in the kernel), so all
> > allocations are routed through malloc...
>--snip--
> Not sure if that is such a bad practice. Maybe it is if you design for
> embedded MMU-less CPUs. But I don't do this. PA is optimized for
> normal, modern Unix/Linux machines, with shared memory. In fact the
> entire memory model PA uses is designed with a good MMU in mind. 

Actually, people are trying to use PulseAudio on non-Intel CPUs in 
non-desktop roles -- outgrowing or outrunning the original design scope.  I 
would very much like to use PulseAudio on the Nokia N810 (Maemo), which 
has an ARM processor.  It does have a working MMU, but...

> That's however how Unix always used to work. In traditional Unix sbrk()
> was called to increase the address space only. Memory was never given
> back to the OS again.

For reasons too baroque to be explained here (or anywhere :-), the Maemo 
task launcher owns all the memory ever used by the web browser and the 
media player, never gives it back, and chokes itself to death.  I find I 
need to reboot my device prophylactically once a day, because the stupid 
task launcher becomes obese.  This is with 128 Mb RAM and a similar swap 
file.  I hope PulseAudio can avoid a similar fate.

> The way memory block allocation in PA works is via a lock-free
> stack. That way both allocation and freeing is fast. It's basically
> just a cmpxchg to allocate or free a block. However to achieve that
> you cannot go to the MMU each time.

Agreed.  But how about a compromise: rather than mapping 64 Mb at once, 
map big but not outrageous blocks: let's say 1 Mb.  If the needs of a 
particular stream are more than 1 Mb (not likely, right?) you give up 
realtime response and use sbrk to get another Mb.  You could even do this 
on speculation: each time real memory is expended, if under 128 Kb of 
mapped space remains, you could wait until the audio is flowing, *then* do 
the sbrk, and (I hope) be back in control before the DMA count runs out.  
Assuming that one has DMA to the DSP or whatever.

> If you want to make PA run well on MMU-less machines there are few
> changes you should be making to PA. Besides disabling memblock
> allocation from the pre-allocated pool in src/pulsecore/memblock.c (by
> forcing that only blocks of type PA_MEMBLOCK_APPENDED are
> allocated) you should disable free lists (i.e. pa_flist_pop() should
> always return NULL), so that we'd always go directly to glibc
> malloc/free. Then you might want to disable the whole shm logic, and
> maybe a few other things.

I'd expect that recycling free blocks would be very valuable in any 
context.  On the other hand, also in any context, it would be valuable to 
occasionally collect garbage.  In other words, find the highest buffer that 
isn't free, adjust upwards according to policy e.g. an even multiple of 1 
Mb, un-chain all free buffers with addresses above the limit, and use brk() 
to shrink the authorized address space.  If the Maemo launcher did this I 
wouldn't need to do Windows-style system administration on my N810 :-(


James F. Carter          Voice 310 825 2897    FAX 310 206 6673
UCLA-Mathnet;  6115 MSA; 520 Portola Plaza; Los Angeles, CA, USA  90095-1555
Email: jimc at math.ucla.edu    http://www.math.ucla.edu/~jimc (q.v. for PGP key)