[HarfBuzz] Harfbuzz-ng and Myanmar

[Behdad Esfahbod]

On 10/26/2009 04:11 AM, Keith Stribley wrote:
> Hi Behdad,

Hi Keith,

> I've been trying out harfbuzz-ng with pango 1.26 and come across a few
> issues with Myanmar Unicode fonts.

Thanks for trying!


> a) Lookup Flags
>
> The Padauk font (http://scripts.sil.org/Padauk) uses mark and mkmk
> positioning, however, this seems to have been disabled. I tracked this
> down to what appears be an over restrictive use of the lookup flags and
> glyph properties. My current understanding is that lookup_flags can have
> a mark attachment type set within the 0xFF00 mask, in which case the
> mark attachment type bits should match with the glyph property bits.
>
> In my test case of ကု the glyph has a property of 0x208 and the lookup
> flags on the mark lookup that should be applied are 0x200. I was able to
> get mark positioning working again with the following patch.
>
> diff --git a/pango/opentype/hb-ot-layout.cc b/pango/opentype/hb-ot-layout.cc
> index 3b6b8da..2c33cab 100644
> --- a/pango/opentype/hb-ot-layout.cc
> +++ b/pango/opentype/hb-ot-layout.cc
> @@ -148,7 +148,7 @@ _hb_ot_layout_check_glyph_property (hb_face_t    *face,
>     /* Not covered, if, for example, glyph class is ligature and
>      * lookup_flags includes LookupFlags::IgnoreLigatures
>      */
> -  if (property&  lookup_flags)
> +  if (0x00FFu&  property&  lookup_flags)
>       return false;
>
>     if (property&  HB_OT_LAYOUT_GLYPH_CLASS_MARK)
>
> I'm not sure if the mask of 0xFF is correct, or whether it needs to be
> tightened further to just the ignore flags.

You're right.  It should only mask to the ignore flags, that is, 0x0E.  Pushed 
to master.


> http://www.thanlwinsoft.org/~keith/tmp/Padauk-noMark.svg
> shows the current harfbuzz-ng behaviour. Blue is ICU rendering, red is
> Harfbuzz-ng.
>
> http://www.thanlwinsoft.org/~keith/tmp/Padauk-maskingLookupFlags.svg
> shows the correct rendering after the above patch has been applied.
>
> b) Order of applying features
>
> The Myanmar3 font uses a mixture of clig and liga. In Uniscribe and ICU
> these lookups are applied in lookup order regardless of which particular
> feature is being used. In pango all the liga lookups are applied before
> the clig lookups in the basic shaper, which results in incorrect
> rendering since some of the clig lookups are expecting the liga lookups
> to have been applied in order. My understanding is that kern, mark and
> mkmk should also be applied together, in lookup order.
>
> The difference in rendering is shown between
> http://www.thanlwinsoft.org/~keith/tmp/Myanmar3-PangoFeatureOrder.svg
> (wrong)
> http://www.thanlwinsoft.org/~keith/tmp/Myanmar3-LookupOrder.svg (correct)
>
> I'm not sure if the relative order of applying lookups in different
> features is specified in the standard, but I would have thought it would
> be better to be consistent with Uniscribe and ICU's implementations. The
> Padauk font gets around this by just using the clig feature for the GSUB
> table lookups.

Again, you are right.  This is planned to be fixed.  Should be done by the end 
of next week (Jonathan Kew and I have a week-long HarfBuzz hacking sprint in 
Toronto).

https://bugzilla.gnome.org/show_bug.cgi?id=439298


> c) A Myanmar shaper
>
> Myanmar3 and Padauk implement reordering using successive clig lookups,
> which is hard to implement, but does allow correct rendering. A while
> ago in the old harfbuzz a Myanmar shaper was written for one
> interpretation of Unicode 4, but this does not work with Unicode 5.1
> implementations and is the reason why Myanmar Unicode 5.1, 5.2 text
> cannot be displayed correctly in QT applications. Fortunately, Pango
> doesn't use that shaper.
>
> Has the process of porting the old harfbuzz shapers to harfbuzz-ng been
> started, perhaps from Jonathan Kew? If so, where can I view it? Am I
> right to assume that this will eventually replace the code in pango/modules?

They will eventually replace code in pango/modules, yes.  Again, should be in 
place by the end of next week.


> I would be interested in writing a new Myanmar shaper to perform the
> necessary reordering required for Unicode 5.2 to make it faster and
> easier to write Myanmar Unicode 5.1/5.2 compliant fonts. I will make
> sure that it doesn't break the existing fonts such as Padauk and
> Myanmar3 which don't require it.

Excellent!  Will keep you posted.


> d) Harfbuzz-ng interface
>
> The SVGs above were generated with a utility that I use to compare
> rendering between ICU, Harfbuzz and Graphite.
> (http://scripts.sil.org/svn-public/graphite/grutils/trunk) I realise
> that hb_shape is currently just a stub, but I was able to implement the
> OT rendering by looking at basic_engine_shape,
> _pango_ot_info_substitute, _pango_ot_info_position and apply_gpos_ltr.
> The use of positions array from hb_buffer_get_glyph_positions was
> probably the least intuitive, the other aspects were fine.

Interesting.  Got code to share?


> In the process of this I started to write some implementations of the
> hb_font_funcs. Should the hb_font_get_contour_point_func_t have an extra
> argument for the point index? Currently it is defined as:

Right.  Added.  I've not hooked that callback (or any of the callbacks for 
that matter) up yet.


> typedef hb_bool_t (*hb_font_get_contour_point_func_t) (hb_font_t *font,
> hb_face_t *face, const void *user_data,
>                                 hb_codepoint_t glyph, hb_position_t *x,
> hb_position_t *y);
>
> What will be the contents of the hb_glyph_metrics_t? I made a guess at:
>
> struct _hb_glyph_metrics_t
> {
>      hb_position_t x;
>      hb_position_t y;
>      hb_position_t x_offset;
>      hb_position_t y_offset;
>      hb_position_t width;
>      hb_position_t height;
> };

Yes, pretty much.

Thanks again, and I look forward to working with you soon.

Cheers,
behdad


> cheers,
> Keith
>
> _______________________________________________
> HarfBuzz mailing list
> HarfBuzz at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/harfbuzz