[pulseaudio-tickets] [PulseAudio] #895: Naive method for mapping between integer and float sample is numrically instable

[PulseAudio]

#895: Naive method for mapping between integer and float sample is numrically
instable
--------------------------------------------------------+-------------------
 Reporter:  datenwolf                                   |       Owner:  lennart
     Type:  enhancement                                 |      Status:  new    
Milestone:                                              |   Component:  core   
 Keywords:  sample format conversion numeric stability  |  
--------------------------------------------------------+-------------------
 The current implementaton for sample rate conversion of PA uses a simple
 scale & offset based approach. While being easy and straightforward to
 implement, this method is not numerically stable as it is not bijective if
 applied on discrete intervals of different or even varying granularity.

 A concise mathematical description is given in this paper:
 [http://kaba.hilvi.org/programming/range/RangeConversion.pdf]

 It is shown that a stable conversion can be achieved by using
 {{{
 int_to_real(i) = clamp(i / (INT_MAX + 0.5), -1, 1);
 real_to_int(r) = clamp(r * (INT_MAX + 0.5) + sgn(r), -INT_MAX, INT_MAX)
 }}}

 An even more in depth text on the problem is [[BR]]
 ''Robert M. Gray, David L. Neuhoff: [[BR]]
 Quantization [[BR]]
 IEEE Transactions on Information Theory, Vol. 44, No. 6, October 1998''

-- 
Ticket URL: <http://pulseaudio.org/ticket/895>
PulseAudio <http://pulseaudio.org/>
The PulseAudio Sound Server