<div dir="ltr">I did some profiling (still with our caching enabled, I'll try again later with it disabled). These are inclusive sample percentages, which actually makes them slightly clunky - I'll try and get some instrumentation timings too.<div><br></div><div>The total percentage of time spent in <font face="monospace, monospace">hb_shape</font> for the first profile is 16.9% (281 samples), of that, 13.9% (231 samples) was spent in <font face="monospace, monospace">_hb_ot_layout_create</font>, with 10.6% (176 samples) spent in the call to <font face="monospace, monospace">OT::Sanitizer<OT::GPOS>::sanitize</font><font face="arial, helvetica, sans-serif">. I noticed that within the call to sanitize, 3.7% (63 samples) was spent calling </font><font face="monospace, monospace">DEBUG_MSG_LEVEL</font><font face="arial, helvetica, sans-serif"> (from </font><font face="monospace, monospace">check_range</font><font face="arial, helvetica, sans-serif"> and </font><font face="monospace, monospace">check_array</font><font face="arial, helvetica, sans-serif">).</font></div><div><br></div><div>I stubbed out the <font face="monospace, monospace">DEBUG_MSG_*</font> macros to see what difference that would make. The total percentage of time spent in <font face="monospace, monospace">hb_shape</font> for the second profile is 11.4% (112 samples), of that, 7.7% (76 samples) was spent in <font face="monospace, monospace">_hb_ot_layout_create</font>, with 4.2% (41 samples) spent in the call to <font face="monospace, monospace">OT::Sanitizer<OT::GPOS>::sanitize</font><font face="arial, helvetica, sans-serif">.</font></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">I wouldn't put too much stock into those figures, as sample based profiling can easily be influenced by human elements when interacting with the UI, however it does let me raise two points:</font></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">1) Why are the </font><font face="monospace, monospace">DEBUG_MSG_*</font><font face="arial, helvetica, sans-serif"> macros not stubbed out in non-debug builds (eg, when </font><font face="monospace, monospace">HB_DEBUG != 0</font><font face="arial, helvetica, sans-serif">)?</font></div><div><font face="arial, helvetica, sans-serif">2) What is </font><font face="monospace, monospace">OT::Sanitizer</font><font face="arial, helvetica, sans-serif"> doing, and could it be skipped without leading to crashes?</font></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">Thanks,</font></div><div><font face="arial, helvetica, sans-serif">Jamie.</font></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 5 October 2015 at 22:31, Jamie Dale <span dir="ltr"><<a href="mailto:jamiedale88+harfbuzz@gmail.com" target="_blank">jamiedale88+harfbuzz@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><font face="monospace, monospace">hb_ft_font_set_flags</font> sounds basically like what I'm already doing using user-data, so yeah, that sounds like it would be fine.<br><div><br></div><div>I'll see if I can get you some stats (I can't remember off the top of my head), but I'd certainly be willing to help try out caching patches to hb-ft.</div><div><br></div><div>The main reason we just went with our own caching is that it already existed, and we still have some font rendering that doesn't go through shaping (and likely never will) so this keeps our font caching unified between our character-based and glyph-based (shaped) font rendering paths. With our FreeType caching applied, I can say that the slowest part is the OT code (I have recollections of something along the lines of <font face="monospace, monospace">OT::Sanitizer</font> showing up on my profiles), and it also seems that <font face="monospace, monospace">hb_shape_plan_create_cached</font> never actually caches (or if it does, it doesn't do it often enough to avoid repeated work - although I'm not sure if it <i>is</i> caching and then I'm just losing that cached data somehow).</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>-Jamie.</div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On 5 October 2015 at 22:20, Behdad Esfahbod <span dir="ltr"><<a href="mailto:behdad.esfahbod@gmail.com" target="_blank">behdad.esfahbod@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Thanks Jamie,<br>
<br>
For loadflags, would it be enough to add hb_ft_font_set_flags() that sets<br>
loadflags, and use that in all places? I'll go ahead and do that, since it's<br>
been long overdue.<br>
<br>
Re caching, do you have data you can share on which parts are the slowest,<br>
etc? Would you be willing to test a couple of patches if I add caches to hb-ft?<br>
<br>
Thanks,<br>
behdad<br>
<span><br>
On 15-10-05 05:02 PM, Jamie Dale wrote:<br>
> Sure, I've attached the font file we're using.<br>
><br>
> Unfortunately, I don't think our caching solution is anything that could be<br>
> easily made HarfBuzz friendly, as it builds upon our existing font caching<br>
> mechanisms (our use case here is real-time games) and uses a font atlas (plus<br>
> a series of granular caches) to avoid repeatedly asking FreeType for the same<br>
> data. It is itself heavily reliant on our engine code, and makes extensive use<br>
> of our custom user-definable composite fonts (which allow users to create<br>
> their own localisable font families). Our composite fonts also allow you to<br>
> specify which hinting method to use for each font face (as this can produce<br>
> nicer results for certain fonts, such as using light hinting with thin fonts),<br>
> which is why we need to preserve this FreeType hinting behaviour with HarfBuzz<br>
> (they're just extra FreeType load flags).<br>
><br>
> -Jamie.<br>
><br>
> On 5 October 2015 at 21:24, Behdad Esfahbod <<a href="mailto:behdad.esfahbod@gmail.com" target="_blank">behdad.esfahbod@gmail.com</a><br>
</span><div><div>> <mailto:<a href="mailto:behdad.esfahbod@gmail.com" target="_blank">behdad.esfahbod@gmail.com</a>>> wrote:<br>
><br>
> I'm not sure what's wrong. Can you send me the font file?<br>
><br>
> Also, I'm interested to hear more about your caching and load-flags stuff. I<br>
> like to upstream them to the extent that makes sense.<br>
><br>
> behdad<br>
><br>
> On 15-10-05 03:32 PM, Jamie Dale wrote:<br>
> > Hey all,<br>
> ><br>
> > In our application we have two text shaping backends; a simple "kerning only"<br>
> > implementation using our existing font rendering code (fast, but can't handle<br>
> > complex languages), and a HarfBuzz based implementation for when we need to<br>
> > support complex languages (although ideally the HarfBuzz implementation should<br>
> > be able to produce mostly equivalent results when compared to our existing<br>
> > font rendering when dealing with languages like English).<br>
> ><br>
> > If you've seen my previous email entitled "Using FreeType load flags", then<br>
> > you'll know I'm using some custom font functions to try and ensure HarfBuzz<br>
> > uses the correct FreeType hinting flags, as well as cache a lot of FreeType<br>
> > calls, and use the correct scale.<br>
> ><br>
> > So far this has worked out really well, however I've noticed that the HarfBuzz<br>
> > implementation isn't applying kerning when shaping English text (probably<br>
> > other languages too).<br>
> ><br>
> > The images below are using the Roboto font, and show the output from each<br>
> > implementation.<br>
> > The top image is using our "kerning only" implementation, and the bottom image<br>
> > is using HarfBuzz. Note that the "T" and "e" characters are further apart in<br>
> > the bottom image.<br>
> > Inline images 2<br>
> > Inline images 1<br>
> ><br>
> > Here's the (slightly cut down) code I'm using to create my HarfBuzz font:<br>
> ><br>
> > hb_font_t* CreateHarfBuzzFont(FT_Face InFace, const uint32 InGlyphFlags,<br>
> const<br>
> > int32 InFontSize, const float InFontScale)<br>
> > {<br>
> > hb_font_t* HarfBuzzFont = nullptr;<br>
> ><br>
> > // Set the character size to render at (needs to be in 1/64 of a "point")<br>
> > FT_Set_Char_Size(InFace, 0, InFontSize * 64, 96, 96);<br>
> ><br>
> > if (InFontScale != 1.0f)<br>
> > {<br>
> > FT_Matrix ScaleMatrix;<br>
> > ScaleMatrix.xy = 0;<br>
> > ScaleMatrix.xx = (FT_Fixed)(InFontScale * 65536);<br>
> > ScaleMatrix.yy = (FT_Fixed)(InFontScale * 65536);<br>
> > ScaleMatrix.yx = 0;<br>
> > FT_Set_Transform(InFace, &ScaleMatrix, nullptr);<br>
> > }<br>
> > else<br>
> > {<br>
> > FT_Set_Transform(InFace, nullptr, nullptr);<br>
> > }<br>
> ><br>
> > {<br>
> > // Create a sub-font from the default FreeType implementation so we can<br>
> > override some font functions to provide low-level caching<br>
> > hb_font_t* HarfBuzzFTFont = hb_ft_font_create(InFace, nullptr);<br>
> > HarfBuzzFont = hb_font_create_sub_font(HarfBuzzFTFont);<br>
> > hb_font_destroy(HarfBuzzFTFont);<br>
> > }<br>
> ><br>
> > hb_font_set_funcs(...); // Sets functions that use the user-data below,<br>
> > implementations are mostly default, but with a caching layer, and usage of<br>
> > FreeType hinting flags<br>
> > hb_font_set_user_data(...); // Ensures we have access to InGlyphFlags<br>
> and the<br>
> > low-level caches<br>
> > hb_font_set_scale(HarfBuzzFont, InFontScale, InFontScale);<br>
> ><br>
> > return HarfBuzzFont;<br>
> > }<br>
> ><br>
> > Am I just missing something simple? I tried calling hb_font_set_ppem but<br>
> that<br>
> > didn't seem to make any difference.<br>
> ><br>
> > Thanks,<br>
> > Jamie.<br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > HarfBuzz mailing list<br>
</div></div>> > <a href="mailto:HarfBuzz@lists.freedesktop.org" target="_blank">HarfBuzz@lists.freedesktop.org</a> <mailto:<a href="mailto:HarfBuzz@lists.freedesktop.org" target="_blank">HarfBuzz@lists.freedesktop.org</a>><br>
> > <a href="http://lists.freedesktop.org/mailman/listinfo/harfbuzz" rel="noreferrer" target="_blank">http://lists.freedesktop.org/mailman/listinfo/harfbuzz</a><br>
> ><br>
><br>
><br>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>