[FriBidi] Invalid UTF-8 for Arabic

[Behnam Esfahbod ZWNJ]

Hi Yoann,

> Behdad, Thanks for the response.

No problam at all, but please note that I'm behNaM, behDaD's brother.


> Your output is what I get from fribidi 0.10.9, but that doesn't do
> Arabic joining. Other than that BOM mark, the Windows output from
> 0.19.1 is correct and matches what other non-Fribidi-based, bidi
> programs do.

I didn't notice you have get the shaping on the string too.  So the
windows output is correct, except the BOM mark.

Seems the BOM mark appears in the place of the first character of the
LAM+ALEF ligature.  This might be a but in the ligature replacement of
the shaping function.

Yep, it's in the CVS HEAD, lib/fribidi-arabic.c
fribidi_shape_arabic_ligature(), which replaces the first char of
ligature with FRIBIDI_CHAR_FILL, which is ZWNBSP/BOM.

Behdad, why don't you use U+FFFF for this purpose?  and why
fribidi_shape_arabic() or fribidi_shape_arabic_ligature() doesn't
clean this CHAR_FILLs?

Behdad, I can fix the problem in CVS if you tell me what's the best
way to fix this.


-Behnam


-- 
    '     بهنام اسفهبد
    '     Behnam Esfahbod
   '
  *  ..   http://behnam.esfahbod.info
 *  `  *
  * o *   http://zwnj.org