[pulseaudio-discuss] Memory management and other issues in the client library

[Lennart Poettering]

On Wed, 03.09.08 08:39, rdiezmail-pulseaudio at yahoo.de (rdiezmail-pulseaudio at yahoo.de) wrote:

> In fact, I guess most users of the PA source code would only be
> interested in the client library. That is, assuming PA manages to
> gain acceptance in the audio app community. App developers have
> little control of which PA daemon version their distribution of
> choice bundles by default. But they will be using the latest version
> of the client library. That's why I think it should be easier for
> developers to separately download and build the client library. I
> don't want to install libsamplerate and the rest just to get the
> latest libpulse version. One would expect to see a separate forum
> just for client library programming issues.

You are reducing PA to its networking audio abilities.

That said I'd be willing to merge a (clean!) patch that adds a
configure switch for disabling everything but the client library.

> But back to the memory management point: the only thing my embedded
> Linux does not have is shared memory support turned on in the
> kernel. And that's fine. You don't need shared memory in the client
> library if you want to connect to the main audio server via
> TCP/IP. In fact, the client library should use shared memory only to
> speak to a local daemon, if available. For TCP/IP connections, it
> shouldn't touch it.

I already offered you to merge a patch that optionally disables all
functionality that only makes sense on MMU, again with a configure
switch. And I already gave you a few hints where to place #ifdefs for
that.

Please understand that I won't cook up a patch for this myself. First
of all I don't have no access to MMU-less machines, secondly it's not
my area of interest. All I can offer you is that I will merge clean
patches which allow you to use it the way you want to use it. But the
patch has to come from *you*. It's Free Software after all! Scratch
your own itches!

> Furthermore, the client library should not implement its own memory
> management. It should give the application the choice of which
> malloc/free to use, like zlib. It should not implement a memory pool
> (imagine if every library implemented its own memory management once
> again). It should return all memory as soon as possible to the C
> library, so that the released memory becomes available to the host
> application. It should not allocate some fixed 64 MB of memory
> upfront, just because this figure has been seen to improve
> performance on the daemon side.

This is nonsense. The fact that we do data transfer via SHM has a big
benefit for us on all machines that provide SHM. Disabling it
generally just because a handfull of people want to run this stuff on
MMU-less machines is ridiculous.

For the malloc() situation: as I already mentioned most libcs allow
you to replace malloc anyway, I don't need to duplicate a vtable for
this in PA. However, I also already offered you to merge a patch that
would add a vtable for the mallocs if it is clean enough and if you
can give me a really good reason.

Then, PA is no zlib. PA wants to exchange some non-trivial amount of
data between clients and the server. We use the best transport for
that -- on a local machine that is SHM. zlib however is not time
critical in any way and is not about IPC. This comparison of yours
stinks.

Also I have no problem with having libraries implement their own
memory management if they have good reasons for doing so. In fact a
lot of libraries do. The apache runtime does it (apr_palloc() and
friends). Glib does (GSlice). The Samba libs do it (talloc). And they
all have good reasons to do it. As do I.

Finally, I already tried to explain that PA allocates address space,
not memory, on all systems that support paged memory. Of course,
that's not true for MMU-less machines. But that's not my area of
interest. And again, I already offered you to merge patches which
allow you to disable the pool based allocation as a compile time
switch. (see above)

> If the client library needs special memory features for real-time
> (like "do not swap out" flags), it should pass some flag to the
> application-provided malloc routine. That's assuming you still have
> any real-time hopes when you connect via TCP/IP.

Oh, that'd be fun. So I'll ask the client app: please allocate me some
memory, but do it lock-free, and in this SHM please. What do you
expect the client app would do with such a request?

Really, this makes no sense. The very reason we have the client API is
for not needing the client programmers to understand the
exceptionally complex logic of the memory block allocation and
transfer and its interfacing with the protocol.

If we could simply outsource all these issues to the client programmer
we wouldn't need the client library at all!

And don't forget the reason why we do this mempool based allocation:
we want lock-free allocation. lock-free algorithms are really
difficult to get right. really fucking difficult.

Lennart

-- 
Lennart Poettering                        Red Hat, Inc.
lennart [at] poettering [dot] net         ICQ# 11060553
http://0pointer.net/lennart/           GnuPG 0x1A015CC4