[gst-devel] float caps

[Andy Wingo]

Hi Benjamin,

This is a discussion we've been through many times. I'll document the
full extent of the problem sometime later, but I need to make it home
now and so this will be necessarily short.

On Mon, 19 Jan 2004, Benjamin Otte wrote:

> However float seems to subscribe to the one-pad-per-channel mentality that
> just doesn't work with GStreamer. For several reasons:
> - having multiple connections between elements easily leads to deadlocks

Not if
 (1) Elements adhere to the frames-per-buffer caps property
 (2) An element does not change the data flow rate, as measured in
     frames/sec, when in a multipad pipeline (impossible unless you set
     things up that way yourself)

> - even if it works it's slow because of all that cothread switching

I use opt. I know you think it's broken, but it's quite fast for my work
and I haven't seen a problem yet. Ignore that scheduler, if you will,
but it's being used. It's also more friendly to the stack, which is
important for guile.

> - it's extremely hard to code with REQUEST and SOMETIMES pads (I reduced
> the float2int code from 530 to 180 lines while adding stuff)

I coded all of that. It was tough, but there were design constraints
there that perhaps you don't share.

> - Nowhere in GStreamer are multiple connections used to represent one
> "real" connection. Try convincing autopluggers that they might need to use
> multiple connections to get stereo output for vorbis.

Float data is a restricted realm that should not be autoplugged. Mono
float is for DSP and analysis only, not for autoplugged media pipelines.

> And as far as I can see in our current elements, noone needs multiple
> pads.

DSP is a lot easier with them. That's why it's that way. Check the l-a-d
archives if you're unconvinced. And even if you're unconvinced and you
want to play with vorbis or whatever, don't mess with this setup. My
synthesizer depends on this behavior.

> I just saw that float2int and alsasink are a hell of a lot more
> complicated then they need to be just because of float.

True.

> So my questions are:
> - Why was that design decision made and is there any reason to keep using
> it?

Search l-a-d. And yes, any kind of pro audio app (mine is one) will need
this functionality. Also check the design of LADSPA, both the standard
and our plugins.

> - If not and we designed a new float format, how should data in it be
> represented? What are the requirements?

Perhaps this is the best way forward. But don't get rid of the old
framework. That would give us 2 caps formats for what really are two
different kinds of data: float-for-dsp and
float-for-autoplugging-media-what-what. And you can allow interleaving
in that format. But please, DON'T MESS WITH THE BEHAVIOR OF ADDER OR
FLOAT/INT OR LADSPA PLUGINS without a full discussion here, on the list.

> - Who is using float audio atm?

Me and iain, I think, prolly steveb as well. Maybe the pakt kids, I
dunno.

Man, I sound grumpy. I guess I am because I have to make this argument
every six months or so. But there are good reasons for it. So please, if
you don't do any DSP work, just leave that area of caps-space alone :-)

Regards,

Wingo