[gst-devel] Decoupled elements and schedulers

[Benjamin Otte]

On Tue, 30 Mar 2004, Martin Soto wrote:

> By the way, should I commit it? It seems ready for more general testing,
> now that it runs my DVD player app correctly, and will allow us to test
> scheduling related core changes (and we *will* need a few more changes,
> that's for sure).
>
Please commit it when you feel it's ready, since it ceartainly won't break
anything.
I committed mine after it worked with Rhythmbox and gst-player.

> I think it is really good that you coded a separate scheduler. If we
> really want to be able to have replaceable schedulers, we need to have a
> few working schedulers to test against.
>
Yeah, I think now that we are both writing schedulers we can figure out
what we want for scheduling.

> > Imagine too, that two different schedulers are used for the two threads,
> > scheduler X for the first thread, your scheduler for the second thread.
>
> Ugh. I was wondering if that was possible, but looking at the existing
> interfaces (or lack thereof) decided to code under the assumption that
> such a thing was not allowed. Probably I'm wrong, but I'd swear that the
> older schedulers would break miserably if you tried that.
>
I certainly want to have the possibility to use different schedulers for
different parts of a pipeline. That's the reason to have different
schedulers in the first place. Different schedulers are supposed to have
different strengths and tradeoffs for speed/feature-completeness and so
on.
I'll test the older schedulers now that you brought that up by putting
some tests into the testsuite that do exactly this.

> But anyway, the point is whether we want to have that option, because,
> if we do, we'll need to define exactly how schedulers are supposed to
> interact.
>
Let's go ahead. :)

> So, let me try my hand at it. This is sort of a primitive spec. Comments
> are welcome, of course:
>
> --
> A scheduler is responsible for managing a number of elements and links,
> where links are defined as a source pad and a sink pad that are linked
> together. All links between elements managed by a scheduler must be
> managed by the same scheduler. It is possible for a scheduler to manage
> a link where only one of the end elements is managed by the scheduler.
> In that case, the other end element must be decoupled.
>
> When a scheduler manages a link, it owns the sched_private pointers in
> both pads of that link.
>
> One important precision is that elements aren't actually scheduled. What
> is scheduled are certain functions provided by an element, namely, the
> loop, chain and get functions. Let's call them the "schedulable"
> functions of an element [if you have a better word for this, please let
> me know ;-)]
>
> A scheduler has two main jobs. The first one is to make sure that data
> pushed on one end of a managed link can be later pulled from the other
> end. The second one is to run the code in the schedulable functions in
> such a way that data keeps flowing in the pipeline. A scheduler can pick
> whatever run order it sees fit, as long as it guarantees that any
> function that is eligible for scheduling will be eventually scheduled.
>
> Scheduling a function means transferring control to the code of the
> function. A scheduler is free to suspend the execution of a schedulable
> function at certain points (see below) before if has returned, and to
> resume the execution at some later arbitrary point in time.
>
> The rules to decide if a function is eligible for scheduling are as
> follows:
>
> R1: Control can be transfered only to functions belonging to elements
> that are in the PLAYING state.
> R2: Loop functions are eligible at anytime, provided you do not break
> R1.
> R3: Chain functions can only be scheduled if the scheduler can provide
> them with an input block.
> R4: Get functions can on only be scheduled if the scheduler can provide
> at least temporary space to store the returned data.
>
Let me try to make this work with decoupled elements. This requires the
definition of "belonging to":
A function belongs to an element if it is attached to the element directly
or to a pad that is a child of the element.
A function belongs to a pad if it is attached to a pad.

R1: Control can be transfered only to functions belonging to elements
    that are in the PLAYING state.
R2: If the element is not decoupled only one of all schedulable functions
    belonging to an element may run at once. This function must have
    finished running before another function may be called.
R3: If the element is decoupled, the same rules as R2 apply per pad, not
    per element. You may not call loop functions for decoupled elements.
R4: Loop functions are eligible at anytime.
R5: Chain functions can only be scheduled if the scheduler can provide
    them with an input block.
R6: Get functions can on only be scheduled if the scheduler can provide
    at least temporary space to store the returned data.

The execution of a function can be suspended [at least?] in the
following cases:

R7: If the function explicitly performs a yield operation, the scheduler
    is free to suspend its execution and give control to other functions.
R8: If the function performs a push, pull, select or clock wait
    operation, the scheduler is free to suspend its execution and give
    control to other functions.
R9: If the function performs a block, the scheduler must interrupt
    scheduling this element (or pad in the decoupled case) until an
    unblock happened.

I'll come to R2 and R3 below. R9 is a future extension that should allow
elements to transfer control to the scheduler instead of interrupting
control flow. The best example for this is the queue. Instead of calling
gst_cond_wait when the queue is full and waiting for the other pad to do a
g_cond_signal, I intend the queue to do a gst_scheduler_block. The other
side of the queue would then of course do a gst_scheduler_unblock which
would allow rescheduling the element.

> This spec of course doesn't imply that you have to register decoupled
> elements. But it seems to me that, in the general case, a scheduler may
> need to know when a decoupled element changes states. Actually, I'd
> rather say both involved schedulers must be informed, but I don't really
> know how to do that. What I'm doing in the fair scheduler is managing
> everything from the side where the decoupled element is registered. This
> assumes, however, that both schedulers are of the same type. We need a
> more general solution, anyway.
>
The way I see it is this:
For normal (that is non-decoupled) elements, you basically schedule the
element. This means only one schedulable function is running per element
and everything a function belonging to the element does happens for the
whole element. This also means that if you have 3 sinkpads, each with
their own chainfunction, if one performs a wait, you may not schedule the
other three chainfunctions in the meantime. The pads are coupled.
In the decoupled case, the element is just a container of pads that can
all be scheduled independantly - they are decoupled. This means every
action a function does (a wait or a block) is only relevant for that pad.
This somehow means (at least for me) that no scheduler can be in control
of the element itself. We have to at least make sure a scheduler in
control of the element is not in control of the links of this element.
(It might work if we changed the behaviour in that patch so that
decoupled elements don't do unsetting the scheduler on links from the
decoupled element, now that I think about it. Would that work for you?)
Long term I'd still prefer if we didn't register decoupled elements with a
scheduler, because that sounds like the right thing to me.

> On the other hand, we can fix the link registering behavior by
> registering (and unregistering) links based on the rules stated before.
> That means, links between two elements in a scheduler are registered
> with that scheduler. Links between a normal and a decoupled element are
> registered with the non-decoupled element's scheduler. Links between two
> decoupled elements should not be allowed.
>
This is definitely correct. The only thing we need to mention is that
linking two decoupled elements' pads can simply be done by putting an
identity in between, so it's not a limitation.

> The one large issue that still remains is clean up. If you abort the
> execution of schedulable functions as the fair schedulers sometimes
> does, you could end up leaking memory, namely, blocks of dynamically
> allocated memory that are only pointed to from the aborted cothread's
> stack. I don't have a general solution for that either. One option I've
> thought of is letting the functions run until they finish, but under
> special conditions. These would be something like all pull operations
> return a "synthetic" EOS event, and all push operations discard the data
> silently. As far as I know, this would bring most schedulable functions
> to a clean termination, but I may be wrong.
>
The theory says elements should not do that. If some elements must do that
we need to invent a way for them to allow that. I don't think any element
does that though, they all use pointers from their element struct. (If
they clean them up correctly when going back to READY or being destroyed,
now that's another question.)

Benjamin