[gst-devel] caps understanding problem

[Benjamin Otte]

On Thu, 20 Jan 2005, Stefan Kost wrote:

> I have a understanding problem with caps
>
> I've got a level element and would like to connect vu-meters to it.
> Therefore I need to find out how many channels of audio are going through it.
> When is it negotiated? AFter everything is connected or only when going to
> playing state?
>
The general rule for negotiation in 0.8 is this:
Negotiate as early as possible, but not earlier.

The long answer:

Negotiation happens when both pads know what caps they can accept.
There are 2 basic types of pads here plus 2 other types which can be
mapped to the first two:
1) Pads that know from the start what caps they can accept. An example
here is sinesrc.
2) Pads that need to inspect the data before knowing. An example here are
srcpads of demuxers like avidemux. Note that this type of pads are
generally source pads, because a sink pad can never inspect any data.
(Type 4 and tee can make that wrong though, but that's another story)
3) Pads that need to inspect devices or similar and look up the supported
caps - think osssink here. Since this happens before any negotiation
attempts, this is pretty much equivalent to point 1.
4) Pads that compute their caps based on the caps of other pads, like
identity for example. Since it's based on those other pads, they're just
proxies of either point 1 or 2.

Going from all the above, pads can report 4 different caps:
- gst_pad_get_template_caps()
Returns the most general caps of a pad, the ones before opening a device
or anything. These are stored in the registry. (For ximagesink, this is
just "video/x-raw-rgb", we don't know the X server resolution yet)
- gst_pad_get_caps ()
Returns the caps that the pad currently thinks it might be able to accept.
In the worst case, these are equal to the template caps. (An unlinked
identity will report "ANY", if you connect it to sinesrc, it'll proxy
sinesrc's caps)
- gst_pad_get_allowed_caps ()
This function is for plugins. It returns an intersection of all caps on a
link that _don't_ belong to the pad - so it's the intersection of
filtercaps and the peer's gst_pad_get_caps (). This function should
probably only be used by pads of type 4) above.
- gst_pad_get_negotiated_caps ()
Once a pad is negotiated, the fixed caps of the link go here. These are
equivalent of the pad's "caps" property.

Until negotiation happens - or after a link has been unnegotiated for
some reason, gst_pad_get_negotiated_caps () returns NULL. When negotiation
has happened, it returns the negotiated pads. Directly after negotiation,
g_object_notify (pad, "caps") is called. So if you connect a signal
handler to "notify::caps", you have the equivalent of an on_negotiate
function.

Negotiation of a link can happen at 2 different points in time (unless
there's some weird element that i'm not aware of that does things
differently):
a) When going from READY to PAUSED, the core checks if both pads can be
negotiated. If so, it negotiates the link, otherwise, the link stays
unnegotiated. This happens when both src and sink are type 1) pads.
b) In the PLAYING state, when the pad has inspected the data and knows
the caps it can accept, it initiates negotiation of the link itself. This
happens by calling gst_pad_try_set_caps (or the pretty much equivalent
gst_pad_set_explicit_caps for pads with explicit caps).


So, this is an overview of how negotiation and caps on pads work. Looking
back at your mail, I guess what you want to do is use a "notify::caps"
handler and only look at gst_pad_get_negotiated_caps().

Benjamin