[gst-devel] caps nego

[Wim Taymans]

On Thu, 2004-07-15 at 16:22, Thomas Vander Stichele wrote:
> I was getting frustrated with my inability to express what doesn't feel
> right about the current caps nego system.
> 
> I attached the result from gst-launch for three pipelines.
> 
> The first one is videotestsrc ! identity ! ffmpegcolorspace ! identity !
> xvimagesink.
> 
> This is a very easy pipeline, anyone can see that the only ones who
> should ever decide about w, h and fps are the src and the sink element,
> and both of them implement a fixation function that also try to fixate
> to the exact same values for them.
> 
> Yet the pipeline ends up fixating to 16x16 at 1 fps.
> 
> I *think* I have some understanding of what goes wrong; some links
> between elements that have no preference for fixating do not fixate,
> hence the core feels it should help in fixating and ends up fixating to
> the lowest int value.
> 
> However, this is a very simple pipeline example that should be fairly
> trivial to fixate.  If our system can't negotiate it correctly, I'd say
> it's a flaw in our system.

Yeah, I'm wasting 70% time on capsnego issues, so there is something
wrong.

Caps nego should be simple, it goes like this:

- negotiation always goes from left to right, ie upstream elements 
initiate the negotiation right before they send the first data buffer.
the element does:

  - query the link for allowed caps (intersection between padtemplate
caps and app filter and getcaps function)
  - loop over the list, fixate with known properties (intersect) and
does try_set_caps. plugins with just one output format do try_set_caps
directly.
  - optionally it could be good to have a can_accept function so that a
format can simply be checked.
 
exceptions:

- a plugin doesn't care about the format, so it doesn't set caps on the
downstream pad. examples filesrc, an element that counts buffers etc..
identity is also such an element, it just proxies the caps.

- an element wants to renegotiate the connection with an upstream
element. ie videosink wants to resize. It calls gst_pad_renegotiate.
Again the upstream filter takes the above mentioned steps to find a
caps. 

Note:

- no automatic negotiation, it's not needed. If an element receives a
buffer without having a required fixed property, it should fire a signal
so that somebody can fixate the caps. 
- If an element does try_set_caps with some non fixed properties one can
fire a signal to let the app fixate them, but it's an app issue.
- An element always has a default format it uses when there are multiple
options. ie osssrc negotiates to a user specified default format, same
for v4lsrc. This should eventually end up in a to-be-written profile
system.
- a caps doesn't have to be fixed for dataflow to be possible. see
filesrc. When nobody cares about the format we're not going to waste CPU
cycles on it. 
- no negotiation is needed in the state changes, only at runtime. our
previous system tried to do automatic negotiation when the intersection
between the pads was fixed. This is too complicated and too fragile.
Better make the element set the caps explicitly. It's a few lines of
code, it's readable and we can all easily debug it when it fails.

> 
> Since the whole capsnego system is too complicated for me to understand
> currently, I'd like to know if this is an actual flaw in our system, a
> bug in plugins, a bug in core, or something else.  I'd also like to know
> what can be done to fix it.  The pipelines we use in the server are
> slightly more complex, and hence we are forced to bruteforce caps on
> every link to actually get somewhere, which is a pain in the ass.
> 

I'm figuring out if we can adjust the current system to a more sane
behaviour or if we should redo caps nego (again). We really need to fix
this asap, I'm thinking in like 2-3 weeks. If anyone is objecting, speak
now :) Above mentioned outline should work as base for the new system,
it was proven to work better (see previous caps system) than the current
system and we're even correcting the errors of the old system.
DirectShow uses a very similar system.

> The other examples exhibit different versions of the same problem, with
> some of them reporting caps on pads that are bogus then get corrected,
> but the end result still ends up wrong.
> 
> Any ideas ?
> 
> 
> Dave/Dina : future TV today ! - http://www.davedina.org/
> <-*- thomas (dot) apestaart (dot) org -*->
> "Would you like my advice ?"
> "That depends. Will I like it ?"
> "Of course you won't ! My advice is based on reality !"
> <-*- thomas (at) apestaart (dot) org -*->
> URGent, best radio on the net - 24/7 ! - http://urgent.fm/
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by BEA Weblogic Workshop
> FREE Java Enterprise J2EE developer tools!
> Get your free copy of BEA WebLogic Workshop 8.1 today.
> http://ads.osdn.com/?ad_id=4721&alloc_id=10040&op=click
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel