[Libreoffice-commits] core.git: external/lpsolve

[Stephan Bergmann]

On 06/18/2014 01:27 PM, Stephan Bergmann wrote:
> commit e7e39d396913b002bdf9f76643f59108d849f392
> Author: Stephan Bergmann <sbergman at redhat.com>
> Date:   Wed Jun 18 13:26:50 2014 +0200
>
>      Avoid undefined out-of-range conversions from double to unsigned char
>
>      Change-Id: I7cf4af81d477865aa0a93c8aa071a8785677c572
>
> diff --git a/external/lpsolve/UnpackedTarball_lpsolve.mk b/external/lpsolve/UnpackedTarball_lpsolve.mk
> index 464fcab..3d17c0e 100644
> --- a/external/lpsolve/UnpackedTarball_lpsolve.mk
> +++ b/external/lpsolve/UnpackedTarball_lpsolve.mk
> @@ -26,6 +26,7 @@ $(eval $(call gb_UnpackedTarball_add_patches,lpsolve,\
>   	external/lpsolve/lp_solve-aix.patch \
>   	external/lpsolve/lp_solve-fixed-warn.patch \
>   	external/lpsolve/lp_solve_5.5.patch \
> +	external/lpsolve/lpsolve-ubsan.patch.0 \
>   ))
>
>   $(eval $(call gb_UnpackedTarball_add_file,lpsolve,lpsolve55/ccc.static,external/lpsolve/ccc.static))
> diff --git a/external/lpsolve/lpsolve-ubsan.patch.0 b/external/lpsolve/lpsolve-ubsan.patch.0
> new file mode 100644
> index 0000000..29e9ad3
> --- /dev/null
> +++ b/external/lpsolve/lpsolve-ubsan.patch.0
> @@ -0,0 +1,11 @@
> +--- lp_pricePSE.c
> ++++ lp_pricePSE.c
> +@@ -145,7 +147,7 @@
> +
> +   /* Store the active/current pricing type */
> +   if(isdual == AUTOMATIC)
> +-    isdual = (MYBOOL) lp->edgeVector[0];
> ++    isdual = lp->edgeVector[0] != 0.0;

As there appears to be no working upstream for lpsolve (at least I 
didn't find anything to report or discuss bugs at 
<https://sourceforge.net/projects/lpsolve/>), lets keep it LO:  The 
above makes me wonder whether the replacement's effect is really what 
had been intended to be the original's effect.  isdual is of type MYBOOL 
aka unsigned char, and lp->edgeVector[0] is of type REAL aka double.

I mean, assuming 8-bit unsigned char, any lp->edgeVector[0] value not in 
the open interval (-1, 256) would have resulted in undefined behavior 
anyway, but even inside that interval, any values in the open intervals 
(-1, 0) and (0, 1) would have resulted in a MYBOOL value of FALSE (aka 
0), and values in the half-open interval [2, 256) would have resulted in 
MYBOOL values different from TRUE (aka 1), where at least AUTOMATIC (aka 
2) and DYNAMIC (aka 4) appear to be relevant.

Maybe somebody who cares about LO's use of lpsolve has an idea.

Stephan

> +   else
> +     lp->edgeVector[0] = isdual;
> +