[Bug 51367] New: Major dwrite:analyzer, dwrite:layout and dwrite:font slowdown when adding (CJK) fonts
https://bugs.winehq.org/show_bug.cgi?id=51367
Bug ID: 51367 Summary: Major dwrite:analyzer, dwrite:layout and dwrite:font slowdown when adding (CJK) fonts Product: Wine Version: 6.10 Hardware: x86-64 OS: Linux Status: NEW Severity: normal Priority: P2 Component: dwrite Assignee: wine-bugs@winehq.org Reporter: fgouget@codeweavers.com Distribution: ---
Created attachment 70223 --> https://bugs.winehq.org/attachment.cgi?id=70223 Command and fonts for Debian's locales
Adding fonts causes major slowdowns in dwrite:analyzer, dwrite:layout and dwrite:font when running in Wine. On my test VM I get the following run times:
Base Full set dwrite:analyzer <1s 50s dwrite:layout ~1s 46s dwrite:font ~2s 85s
On the TestBot's debiant2 VM dwrite:font actually runs in about 118 seconds with the full set of fonts; which is why it times out about 6% of the time:
https://test.winehq.org/data/patterns.html#dwrite:font
The base set of font packages is: fonts-dejavu-core fonts-freefont-ttf fonts-liberation fonts-quicksand
The problem is that the base set produces unreadable text when running in a Chinese, Japanese or Korean locale: all translated strings are replaced with a set of black squares (surprisingly ar_MA, he_IL, hi_HI are ok). That does not seem to impact the test results but it does not feel right either. At a minimum if Debian says a given set of fonts are needed for a locale we should test the said locale with that set of fonts.
So the full set of fonts was created by installing the font packages for all of Debian's locale task packages and this is what debiant2 has. See apt_wine_locales() in wt-install-dev [1]. For reference I attached the corresponding set of packages.
The run time increase is not concentrated in one specific test_xxx() function that would iterate over all the fonts. All test_xxx() functions take longer to run. Even the test_CreateFontFromLOGFONT() run time increased to ~17s (iirc). Yet it does not look like it's doing all that much.
To speed things up I modified dwrite:font to use multiple threads using the same approach as d3d10core:d3d10core. However even with 8 threads that only gained about 4 seconds out of 118s: each individual test function just took longer.
Is that because of lots of global locking? Or because each test adds/removes fonts forcing all the other threads to do many rescans? Are all fonts reloaded and analyzed despite most fonts not having changed?
[1] https://github.com/fgouget/wt-daily/blob/master/wt-install-dev#L556
https://bugs.winehq.org/show_bug.cgi?id=51367
François Gouget fgouget@codeweavers.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Keywords| |source, testcase Summary|Major dwrite:analyzer, |Major dwrite:analyzer, |dwrite:layout and |dwrite:layout and |dwrite:font slowdown when |dwrite:font slowdown when |adding (CJK) fonts |adding fonts
--- Comment #1 from François Gouget fgouget@codeweavers.com --- Maybe installing all these font packages is a bit much so I tried to identify which font packages are causing the biggest run time increase:
2.5s Base set of fonts. 3.1s +0.6s +fonts-not-cjk (fonts-noto recommends) -> sufficient to fix the menus in Notepad 3.6s +0.5s +fonts-noto-cjk-extra (fonts-noto recommends) 11.3s +7.7s +fonts-noto-core (fonts-noto depends) 54.9s +43.6s +fonts-noto-extra (fonts-noto recommends) 59.7s +4.8s +fonts-noto-ui-core (fonts-noto recommends) 76.3s +16.6s +fonts-noto-ui-extra (fonts-noto recommends) 76.3s +0.0s +fonts-noto-mono (fonts-noto recommends) 78.0s +1.7s +fonts-noto-unhinted (fonts-noto recommends)
2.5s Base set of fonts. 3.0s +0.5s +fonts-lohit-{beng-bengali,deva,gujr,guru,knda,mlym,taml,telu} 3.3s +0.3s +fonts-nanum 9.4s +6.1s +fonts-{bpg-georgian,dzongkha,farsiweb,kacst,khmeros,sil-abyssinica,sil-andika,symbola,ukij-uyghur,unikurdweb,vlgothic} 9.7s +0.3s +fonts-ipafont
So fonts-noto-extra and fonts-noto-ui-extra are the most troublesome font packages. Removing them would reduce the run time to about 20 seconds on my box or a 4x speedup (so about 30 s on debiant2). They could conceivably be blacklisted by wt-install-dev.
https://bugs.winehq.org/show_bug.cgi?id=51367
--- Comment #2 from François Gouget fgouget@codeweavers.com --- Created attachment 70228 --> https://bugs.winehq.org/attachment.cgi?id=70228 Use multiple threads in dwrite:font
For reference I attached a set of three patches to test and debug multi-thread support in dwrite:font.
https://bugs.winehq.org/show_bug.cgi?id=51367
--- Comment #3 from François Gouget fgouget@codeweavers.com --- Created attachment 70271 --> https://bugs.winehq.org/attachment.cgi?id=70271 Notepad in a Japanese locale with missing fonts
The screenshot shows the 'black squares' issue mentioned in the first comment. To reproduce: $ locale -a | grep ja_JP ja_JP.utf8 $ LANG=ja_JP.UTF-8 ./wine notepad
Installing fonts-droid-fallback or fonts-noto-cjk or fonts-ipafont fixes the display.
https://bugs.winehq.org/show_bug.cgi?id=51367
--- Comment #4 from Nikolay Sivov bunglehead@gmail.com --- (In reply to François Gouget from comment #0)
Created attachment 70223 [details] Command and fonts for Debian's locales
Adding fonts causes major slowdowns in dwrite:analyzer, dwrite:layout and dwrite:font when running in Wine. On my test VM I get the following run times:
Is that because of lots of global locking?
There is a global lock, yes. I don't know how much it contributes.
Or because each test adds/removes fonts forcing all the other threads to do many rescans? Are all fonts reloaded and analyzed despite most fonts not having changed?
Pretty much. There is no caching of any kind, everything listed in Fonts registry key is loaded once you create a shared font factory. Many tests intentionally don't share a factory to keep them independent.
The main reason test times depend on a number of fonts is that some tests iterate over whole collection, and test usually first of each family. I'd like to keep that because it gives a best coverage, and I was able to fix multiple problems that are hard to spot otherwise, mostly for global font properties related to table data interpretation.
Productive thing to do is to analyze which parts are particularly slow. On Windows I believe font file access is optimized by caching things at single system process level, dedicated for such font data caching. That goes back to WPF 3 I believe. It's possible we could get these times improved without such complications. For example when building default collection it currently scans a list of files for every new font, so that loop runs n*n times for n files (it's probably avoidable by collecting unique font file paths first by making enumerator a bit smarter). I don't know if that will have a noticeable impact though. Another thing is to use the global lock less, but keeping data closer to dwrite instances, rather than freetype instances.
For now, the easiest way is to limit a number of fonts so it does not timeout.
https://bugs.winehq.org/show_bug.cgi?id=51367
Jactry Zeng jactry92@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |jactry92@gmail.com
-
WineHQ Bugzilla