The practical problem is observed in Cosmoteer game which fails to display some Chineese characters (when Chineese language is selected for the game UI).
That happens with the specific font shipped with the game (play_msyh.ttf, play_msyh_bold.ttf), replacing it with Wine font avoids the problem. The rendering issue is present in the glyph image we get from freetype library. Opening any of these fonts with ftview program from FreeType demo programs immediately shows the glitches for some Chineese characters. The font is correctly displayed on Windows (as well as rendering the problematic glyphs goes without visual glitches). I am not attaching those fonts here because apparently those are copyrighted materials. The problem is only there when hinting is enabled, disabling it in ftview with 'h' or in Wine by passing FT_LOAD_NO_HINTING (or FT_LOAD_FORCE_AUTOHINT) avoids the problem.
Internally FreeType library is getting errors when interpreting hinting bytecode for those glyphs (various errors, e. g., Invalid_Reference or Stack_Overflow). While those errors may be either FreeType bugs or font bugs, such errors are possible in TTF fonts even without FreeType bugs and both FreeType and Wine already mind such a possibility and trying to handle that, so I guess we are interested to do it correctly (regardless of whether specific problem with the font is a font bug or freetype bug).
freetype_get_glyph_outline() already minds possible bugs related to hinting, if FT_Load_Glyph failed it tries to call that with FT_LOAD_NO_HINTING flag. The problem is that freetype library doesn't return any error (at least now, maybe that changed since the time this Wine code was first introduced) from interpreting hinting bytecode if FT_LOAD_PEDANTIC flag is not specified. While the glyph may be affected by partially executed bytecode which is then aborted by the error. This doesn't look like an accidental bug, it is a documented behaviour [1]: "... In particular, errors from the TrueType bytecode engine are not passed to the application if this flag is not set; this might result in partially hinted or distorted glyphs in case a glyph's bytecode is buggy."
So I guess setting the flag on the first attempt and then falling back to no hinting / no pedantic checking is the right thing to do (and it fixes the issue).
1. https://freetype.org/freetype2/docs/reference/ft2-glyph_retrieval.html#ft_lo...