[Bug 755301] audioconvert: Integer->Float conversion creates values slightly smaller than -1.0

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=755301

--- Comment #17 from sebastien.fenet at yahoo.fr ---
Ok, I see. In my opinion, it is better to work with 64 bits float as
intermediate format.

Anyway, I agree that 16 bits -> 32 bits is lossless but you don't have the same
problem. 

Look what happens if I work directly with Int16... I take my signal and I
divide it by the (assymetric) max value of Int16: m1 = 2**(16-1). If I had
chosen the other (bad) option, I would have divided by m2 = 2**(16-1)-1. You
can easily check that the 32bits-floating-point representation of m1 and m2 is
not the same (this is basically because we go from 16 bits resolved integers to
32 bits resolved floating point numbers).

On the other hand, when you do the same with Int32, you find two integers m1
and m2 that are too close to be resolved when switching to float32.

However, there is something that puzzles me in what you described. Basically,
you told us that the current dividing factor is somehow: float32(max_Int32-1).
Then, you observed that float32(max_Int32-1)=float32(max_Int32). So, if I sum
up, GStreamer's current code divides by float32(max_Int32). What I really don't
understand is how your Int32 encoded signal divided by this float32(max_Int32)
can be greater than 1... Any clue on this or did I misunderstand something?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.