[gst-devel] cothreads problem on Solaris

[Wim Taymans]

On Tue, 2003-01-21 at 18:26, Brian Cameron wrote:
> Gstreamer cothreads gurus:
> I feel pretty over-my-head in terms of understanding the use of cothreads
> in gstreamer, but I'm hoping that someone can help me understand what is
> going on, and hopefully debug the problem. 
> 
Hi,

I don't have enough Solaris knowledge to give a solution but I'll try to
explain how this works, others can correct me or fill in the blanks
(omega, ds, ...)

We're basically implementing makecontext/swapcontext on top of
linuxthreads. We need to implement something ourselves as tests have
shown that pthreads (linuxthreads) and make/swapcontext don't work too
well together for various reasons.

The idea is to put each element (roughly) in a cothread or a userspace
scheduled thread. When a buffer is passed between two elements, we
switch to the other element to make it process the buffer. We can do
this fairly efficiently as we know exactly when to pass control over to
peer elements. We also want to use the cothreads on top of pthreads to
allow a user<->kernel space scheduling policy.

Since each element needs to have its own piece of stackspace, we need to
allocate some space for it. After a lot of experimenting we figured out
that we cannot malloc our own piece of memory because of the way
linuxthreads use the stack pointer to find the thread struct. We
therefore subdivide the existing stack (of the current thread) into N
pieces of memory, each of the cothreads occupy this piece of stackspace.
With a stackspace of 2MB and a default max of 16 cothreads, each
cothread gets a 128K stackspace. Most of the magic for this can be found
in cothread_context_init() which is called right after creating the
thread; the current stack pointer is used to calculate the top of the
stack (which is alligned on STACK_SIZE boundaries).

Then a cothread_context struct is created (also called cothread 0). In 
cothread_create(), a new cothread is created in an empty 128K slot on
the stack, we also mmap that piece of stack memory to make it writable
(and so also executable on x86). pointers to all this info is stored in
a cothread_state struct, along with some flags to mark the state of the
cothread (running, destroyed, ...).

Switching cothreads then goes like this: We save the current stack
context with a setjmp, for a cothread that has not yet been started, we
make the stack pointer point to the slot in the thread stack (using a
few lines of assembler code) and jump to the entry point of the
cothread. Switching to another cothread will setjmp the current context
before performing a longjump on the saved jmp_buf or before setting up
the stack+jump to entry point (for new cothreads).
The switching continues 'till control eventually returns to the 0th
cothread, at which point, control is handed back to the app that
performed a _iterate() on the scheduler.

There is another versions of a cothread implementation that works on top
of linuxthreads (libs/ext/cothreads), largely based on pth.

That said, I think the more portable solution would be to use
makecontext/swapcontext (which apparently doesn't work too well with
linuxthreads, but might work on other pthread implementation). Another
option is to not use any cothreads at all, but use the default stack to
save the element state while switching between them. An implementation
of that scheduler can be found in the optimal scheduler, which is quite
capable but not as optimal under certain conditions (run apps with
--gst-scheduler=opt). Another option would be to create an all pthread
based scheduler that would put each element into its own pthread, using
mutexes and cond vars to pass buffers between elements.

I hope this makes some sense, going to get some sleep now :)

Wim

> Thanks!
> 
> Brian
> 
> ______________________________________________________________________

-- 
Wim Taymans <wim.taymans at chello.be>