[gst-devel] 0.9 proposals

[David Schleef]

On Thu, Dec 02, 2004 at 03:39:55PM +0100, Martin Soto wrote:
> What my experience so far tells me, is that an intermediate solution may
> be the answer. This solution would use a mixture of threads and
> cothreads to achieve an optimum, depending on a number of factors, like
> whether you are in an SMP machine, how efficient the available thread
> and cothread implementations are (absolutely and with respect to each
> other), which latency requirements you have, and, even, whether the
> particular elements in your pipeline can or cannot avoid blocking.

That's an intermediate solution?

One of the goals is to decrease the complexity of the core, so that
it's more approachable by more people.  Using two threading models
simultaneously doesn't seem to be the right direction.

> A first comment: I know that speaking of a mixture of threads and
> cothreads makes some people here shiver. This combination used to cause
> lots of problems before, at least under Linux. Well, I wrote a Pth
> implementation of GStreamer's cothread API, and I'm happily using it
> right now with Seamless (which is certainly threaded).  The trick is to
> use the New Posix Threads Library (NPTL) at the same time.

The problems with this:
 - Not everyone uses NPTL.
 - Not everyone uses Linux.
 - Pth is not nearly as portable and bug-free as one would like.  (Such
   as native Win32)
 - NPTL is (allegedly) faster than Pth.

> This alone
> may help a lot in terms of performance, because, AFAIK, every time you
> lock or unlock a mutex, or wait on, or signal a condition variable, you
> perform at least one system call.

Who cares?  Making the gstreamer core faster or slower by a factor
of two is mostly irrelevant in the grand scheme of things.  I'd
*much* prefer to have a system that everyone can understand and
bug fix with an overhead of 6% than a complicated system at 3%.
The time that I save from fixing gstreamer bugs I will spend on
liboil, which can have a direct and massive affect on applications.
We can always optimize later, *once the bottlenecks are known*.

I will entertain no thoughts that make the core more complicated
(or prevent it from getting less complicated) due to hypothetical
speed increases.  (Actually, I probably will, but please stop me
when I do.)

> - The drawback of cothreads is that blocking operations block the thread
> (and all of its cothreads) anyway.

Yes.  This is, of course, its biggest limitation.  That, and the
related problem that you can't interchange cothreads and threads.

> The other necessary change is to add a way for elements to tell the
> scheduler that they may block.

Or, tell the scheduler how to perform the blocking, e.g., wait on a
file descriptor or lock.  These are essentially the only ways to
block.



dave...