[PATCH][RESEND] xkb: Allocate size_syms correctly when width of a type increases

[Peter Hutterer]

On Tue, May 29, 2012 at 10:17:50AM +0530, Siddhesh Poyarekar wrote:
> The current code seems to skip syms with width less than
> type->num_levels when calculating the total size for the new
> size_syms. This leads to less space being allocated than necessary
> during the next phase, which is to copy over the syms to the new
> location. This results in an overflow leading to a crash.
> 
> Signed-off-by: Siddhesh Poyarekar <siddhesh.poyarekar at gmail.com>
> ---
>  xkb/XKBMAlloc.c |    6 ++++--
>  1 files changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/xkb/XKBMAlloc.c b/xkb/XKBMAlloc.c
> index 645e905..3ffd5da 100644
> --- a/xkb/XKBMAlloc.c
> +++ b/xkb/XKBMAlloc.c
> @@ -375,8 +375,10 @@ XkbResizeKeyType(XkbDescPtr xkb,
>          nResize = 0;
>          for (nTotal = 1, i = xkb->min_key_code; i <= xkb->max_key_code; i++) {
>              width = XkbKeyGroupsWidth(xkb, i);
> -            if (width < type->num_levels)
> +            if (width < type->num_levels || width >= new_num_lvls) {
> +                nTotal += XkbKeyNumSyms(xkb,i);
>                  continue;
> +            }
>              for (match = 0, g = XkbKeyNumGroups(xkb, i) - 1;
>                   (g >= 0) && (!match); g--) {
>                  if (XkbKeyKeyTypeIndex(xkb, i, g) == type_ndx) {
> @@ -384,7 +386,7 @@ XkbResizeKeyType(XkbDescPtr xkb,
>                      match = 1;
>                  }
>              }
> -            if ((!match) || (width >= new_num_lvls))
> +            if (!match)
>                  nTotal += XkbKeyNumSyms(xkb, i);
>              else {
>                  nTotal += XkbKeyNumGroups(xkb, i) * new_num_lvls;
> -- 
> 
> This problem is reproducible on RHEL-5 by using XDMCP query to connect to
> a RHEL-6 server since the keymaps generated are different on RHEL-5 and
> RHEL-6 (and hence the need to adjust the keymap). The code around this
> has not changed much since RHEL-5 so I believe this fix should be
> relevant upstream too.

thanks, merged and will be upstream soon.

Cheers,
  Peter