[Libreoffice-bugs] [Bug 32862] Improve precision of function MOD() for big numbers (7^19 mod 13 is 6 not 7)

[bugzilla-daemon at bugs.documentfoundation.org]

https://bugs.documentfoundation.org/show_bug.cgi?id=32862

b. <newbie-02 at gmx.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |newbie-02 at gmx.de

--- Comment #6 from b. <newbie-02 at gmx.de> ---
calc calculates correctly - in this case and within it's limitations, 

the 'double' representation of 7^19 is 11398895185373144,00 you can 'see' the
digits hidden by calcs 'Standard' display format 1,13988951853731E+016
by '=RAWSUBTRACT(7^19;1,13988E+016)'

that's off by 1 from the correct value 11398895185373143, double-floats have a
granularity of '2' in this range, thus the result of the mod calculation is off
too, that's correct for calc, but not for users, 

to regain some harmony between the expectations of users - who think a computer
for several thousand bucks should be able to calculate better than they
themself - and the meager abilities of calc and ex$el, many ways would be
possible, two come to my mind immediately: 

- calculation results should be provided with information how accurate - or
inaccurate! - they are, and where one has to expect that the inaccuracy
violates the expectations of users, a warning should be given! 

- tasks for which in principle exact results are possible but not directly
reachable in fp-math should be calculated 'intelligently decomposed', e.g. in
extension of @Regina's approach in c#2 one could calculate '=mod(7^19;13)' as
'=mod(7^10;13) * mod(7^9;13)' - and take the whole again mod-13 as the product
might be >=13, so '=mod(mod(7^10;13) * mod(7^9;13);13)',  that holds, 

to recognize the necessity of such decompositions and to initiate them
intelligently is - automated - not trivial, but also not witchcraft, 

1st step i'd propose: each mathematical procedure is preceeded by a
'range-check' and if the operands exceed or fall below the range for which
exact results are possible a warning is issued, 

2nd step: after that 'calculation tricks' can be added bit by bit which extend
the capabilities of the procedures and are activated in case of 'off-range'
instead of the warning, 

the actual solution in calc to compute and display '0' for such tasks is ...
suboptimal ... user take that as a result instead of a warning

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice-bugs/attachments/20201216/c02c3354/attachment.htm>