LO calc formulas

[Noel Grandin]

On Thu, 10 Dec 2020 at 01:25, Dante Doménech <dante19031999 at gmail.com>
wrote:

> I-ll implement a second order kahan algorithm. Third order would be
> overkill ( I don't think anyone is gonna use it for high precision
> calculus  ).
> But it's not only sum and average, variance (stats) and any other formula
> with sums is affected. That's why I need time to track it.
> And by the way, what about using long double for intermediate steps, to
> improve precision. For what I know glibc or whatever uses gcc on fedora
> uses it,
>


Note that calc is, in general, quite performance sensitive in that people
care about how long it takes to calculate large spreadsheets (and some of
them can get very large indeed).

So 128-bit floating point would probably be tricky since CPU largely don't
have hardware support for direction operations on that.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice/attachments/20201210/774f097d/attachment.htm>