[gst-devel] float caps

[in7y118 at public.uni-hamburg.de]

Thanks for your answer. Looking at this and shooting from my hip, how about 
this:
- Since every dsp element needs mono float, it'll just set the channels 
property to 1. Now since this is properly interleaved float audio, we can just 
expand it to more channels and use it in consumer float.
- If we allow the buffer-frames property to be 0 to mean undefined, dsp 
elements could advertise [1 - MAX] in there templates if they want it fixed or 
[0, MAX] if they don't care or even 0 if they require unfixed. As a bonus the 
core would even automatically fixate to undefined behaviour.
- If we split int2float and float2int each into two plugins, one doing channel 
splitting/merging, the other doing the actual conversion, we could use 
float2int and int2float with autoplugging. That would also solve the current 
problems for the people that have been reporting them with int2float ! 
float2int not working.

Another option would be to make multichannel float be not interleaved but in 
succession in the same buffer. This would still be exactly the same for 1 
channel, would allow simpler 1-to-n channel conversions but int2float would be 
harder.
And the third option would be to do 1-channel-per-buffer and push one channel 
after another in multiple buffers. This would still be the same for 1-channel 
float but would make multichannel elements a bit harder. However it would get 
rid of some memcpys when splitting/merging.


And yet another thing that moves us closer to conversion hell would be to 
invent another float format.

Hm, what do you think?

Benjamin

PS: I'm not going to touch your code. In the worst case I'll do the actual 
float to int conversion inside vorbisdec myself (just like vorbisfile does 
currently). But since this problem came up for the equalizer, too I thought 
it'd be better to just simplify the float approach.


Quoting Andy Wingo <wingo at pobox.com>:

> Hi Benjamin,
> 
> This is a discussion we've been through many times. I'll document the
> full extent of the problem sometime later, but I need to make it home
> now and so this will be necessarily short.
> 
> On Mon, 19 Jan 2004, Benjamin Otte wrote:
> 
> > However float seems to subscribe to the one-pad-per-channel mentality that
> > just doesn't work with GStreamer. For several reasons:
> > - having multiple connections between elements easily leads to deadlocks
> 
> Not if
>  (1) Elements adhere to the frames-per-buffer caps property
>  (2) An element does not change the data flow rate, as measured in
>      frames/sec, when in a multipad pipeline (impossible unless you set
>      things up that way yourself)
> 
> > - even if it works it's slow because of all that cothread switching
> 
> I use opt. I know you think it's broken, but it's quite fast for my work
> and I haven't seen a problem yet. Ignore that scheduler, if you will,
> but it's being used. It's also more friendly to the stack, which is
> important for guile.
> 
> > - it's extremely hard to code with REQUEST and SOMETIMES pads (I reduced
> > the float2int code from 530 to 180 lines while adding stuff)
> 
> I coded all of that. It was tough, but there were design constraints
> there that perhaps you don't share.
> 
> > - Nowhere in GStreamer are multiple connections used to represent one
> > "real" connection. Try convincing autopluggers that they might need to use
> > multiple connections to get stereo output for vorbis.
> 
> Float data is a restricted realm that should not be autoplugged. Mono
> float is for DSP and analysis only, not for autoplugged media pipelines.
> 
> > And as far as I can see in our current elements, noone needs multiple
> > pads.
> 
> DSP is a lot easier with them. That's why it's that way. Check the l-a-d
> archives if you're unconvinced. And even if you're unconvinced and you
> want to play with vorbis or whatever, don't mess with this setup. My
> synthesizer depends on this behavior.
> 
> > I just saw that float2int and alsasink are a hell of a lot more
> > complicated then they need to be just because of float.
> 
> True.
> 
> > So my questions are:
> > - Why was that design decision made and is there any reason to keep using
> > it?
> 
> Search l-a-d. And yes, any kind of pro audio app (mine is one) will need
> this functionality. Also check the design of LADSPA, both the standard
> and our plugins.
> 
> > - If not and we designed a new float format, how should data in it be
> > represented? What are the requirements?
> 
> Perhaps this is the best way forward. But don't get rid of the old
> framework. That would give us 2 caps formats for what really are two
> different kinds of data: float-for-dsp and
> float-for-autoplugging-media-what-what. And you can allow interleaving
> in that format. But please, DON'T MESS WITH THE BEHAVIOR OF ADDER OR
> FLOAT/INT OR LADSPA PLUGINS without a full discussion here, on the list.
> 
> > - Who is using float audio atm?
> 
> Me and iain, I think, prolly steveb as well. Maybe the pakt kids, I
> dunno.
> 
> Man, I sound grumpy. I guess I am because I have to make this argument
> every six months or so. But there are good reasons for it. So please, if
> you don't do any DSP work, just leave that area of caps-space alone :-)
> 
> Regards,
> 
> Wingo
> 
> 
> -------------------------------------------------------
> The SF.Net email is sponsored by EclipseCon 2004
> Premiere Conference on Open Tools Development and Integration
> See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> http://www.eclipsecon.org/osdn
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>