[Bug 53929] New: ExttextOut does not support output surrogate pair（ex:U+1F34A -> 0xd83c,0xDF4A）

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

Bug ID: 53929
           Summary: ExttextOut does not support output surrogate
                    pair（ex:U+1F34A -> 0xd83c,0xDF4A）
           Product: Wine
           Version: 7.12
          Hardware: x86
                OS: Linux
            Status: UNCONFIRMED
          Severity: blocker
          Priority: P2
         Component: gdi32
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

Wine will treat a four-byte unicode as two two-byte unicode output, resulting
in the inability to display characters correctly.

At the same time, wine will treat four-byte unicode as two two-byte unicode to
calculate the width.

The latest version of wine does not work too.

[Bug 53929] ExttextOut does not support output surrogate pair（ex:U+1F34A -> 0xd83c,0xDF4A）

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

[e-mail] changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P2                          |P1

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

Zeb Figura  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|blocker                     |minor
            Summary|ExttextOut does not support |ExtTextOut() does not
                   |output surrogate            |support surrogate pairs
                   |pair（ex:U+1F34A ->          |
                   |0xd83c,0xDF4A）              |

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

Fabian Maurer  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #1 from Fabian Maurer  ---
Windows does that properly? I guess a testcase would be nice, did you run into
that using a certain program?

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #2 from [e-mail] ---
(In reply to Fabian Maurer from comment #1)
> Windows does that properly? I guess a testcase would be nice, did you run
> into that using a certain program?

There is no such problem on Windows. I was in the East Eighth District
yesterday. The time you replied was early morning at 01:30am, so I couldn't
reply your message in time.I'm sorry.

I wrote a demo of a windows program, which is used to input an emoji ????
(U+1F34A), and the Textout API is used here to output characters (Textout
directly calls ExtTextout in wine), and ExtTextout accepts WCHAR * characters
String (0xD83C, 0xDF4A), here involves "surrogate_pai", which uses four bytes
to represent a character, but I looked at the implementation of ExtTextout, he
only accepts two bytes each time, so he thinks that a four-byte is two Two
bytes, so ???? cannot be output.

Because wine thinks that all unicode is two bytes（0xD83C） to represent a
character, so there will be many other problems here, such as
GetTextExtentPoint32W will get the wrong character width. At the same time, if
I use "宋体" as the output font, ???? this character has no corresponding glyph
in the font, windows will use other fonts for output（font fallback）, and
accurately calculate the font width (for line breaks, etc.), but wine will use
the font you set (宋体) to calculate the width, I haven't written a corresponding
demo about this issue

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #3 from [e-mail] ---
Created attachment 73496
  --> https://bugs.winehq.org/attachment.cgi?id=73496
demo ues Textout API to output ????

???? are output as two boxes, and  output as an icon on windows

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #4 from [e-mail] ---
(In reply to 399989567 from comment #3)
> Created attachment 73496 [details]
> demo ues Textout API to output ????
>
> ???? are output as two boxes, and  output as an icon on windows

???? is U+1F34A

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #5 from [e-mail] ---
Created attachment 73497
  --> https://bugs.winehq.org/attachment.cgi?id=73497
DemoSourceCode

main Source Code:

static WCHAR orange[] = {0xd83c,0xdf4a,0};
SelectObject(hdc,宋体);
TextOut(hdc,x,y,orange,wcslen(orange));

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #6 from Fabian Maurer  ---
Created attachment 73502
  --> https://bugs.winehq.org/attachment.cgi?id=73502
Screenshot of my Win7 VM

Sorry, I can't get it to work on Windows either. Can you please tell me what I
need to do to work with that? I probably need a font, I tried the one at
https://fonts.google.com/noto/specimen/Noto+Sans+SC but that didn't help.

Also, a screenshot would probably be good.

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #7 from Nikolay Sivov  ---
I don't see U+1F34A being supported in Noto Sans CJK SC. Might be worth trying
e.g. U+1F200.

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #8 from [e-mail] ---
I compiled a release version of orange.exe, you can double-click to run it
directly on Windows 10。The result of the operation is shown in Figure 1, and
then the same exe, the result of running on wine is shown in Figure 2, it is
divided into two characters for output

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #9 from Rafał Mużyło  ---
Just a guess on my part, but the first compatible google result seems to be
Noto Emoji - comment 2 says explicitly the example is about emojis (they're -
after all - most prominent non-BMP Unicode group in common use).

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #10 from [e-mail] ---
Created attachment 73503
  --> https://bugs.winehq.org/attachment.cgi?id=73503
orange.exe

run on windows10

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #11 from [e-mail] ---
Created attachment 73504
  --> https://bugs.winehq.org/attachment.cgi?id=73504
Figure1 on windows10

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #12 from [e-mail] ---
Created attachment 73505
  --> https://bugs.winehq.org/attachment.cgi?id=73505
Figure 2 on debian

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #13 from [e-mail] ---
(In reply to Fabian Maurer from comment #6)
> Created attachment 73502 [details]
> Screenshot of my Win7 VM
>
> Sorry, I can't get it to work on Windows either. Can you please tell me what
> I need to do to work with that? I probably need a font, I tried the one at
> https://fonts.google.com/noto/specimen/Noto+Sans+SC but that didn't help.
>
> Also, a screenshot would probably be good.

you can use the font Noto Emoji Regular to show the text

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #14 from [e-mail] ---
Created attachment 73506
  --> https://bugs.winehq.org/attachment.cgi?id=73506
Noto Emoji

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #15 from [e-mail] ---
(In reply to Nikolay Sivov from comment #7)
> I don't see U+1F34A being supported in Noto Sans CJK SC. Might be worth
> trying e.g. U+1F200.

You are right,  U+1F34A are not supported in Noto Sans CJK SC.

But I checked the source code, the problem is not that there is no
corresponding glyph of U+1F34A in Noto Sans CJK SC, but that the str accepted
by ExtTextOutW is in UTF-16 format, so U+1F34A is converted into 0xD83C and
0xDF4A, a loop is used to traverse the str string, so U+1F34A is considered as
two separate characters.

like:
> ExtTextOut get WCHAR *str -> {0xd83c,0xdf4a,0}
> for(int i = 0; i < count; i++)
>     find_glyph_from_font(font,str[i])

By the way, Using the ScriptString API in wine, the problem can be solved very
well, but I don't think this modification is perfect.

We also need to get GetTextExtentPoint32W to return the correct character width
when dealing with surrogate pairs

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

Fabian Maurer  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
             Status|UNCONFIRMED                 |NEW

--- Comment #16 from Fabian Maurer  ---
Ah, so you do need Win10, Win7 doesn't work.

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #17 from [e-mail] ---
Created attachment 73509
  --> https://bugs.winehq.org/attachment.cgi?id=73509
path maybe help

Using my path can solve the problem, but this workaround is rude, doesn't solve
the problem at the source, and introduces a lot of problems including length
calculation, so I come to ask for help

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #18 from [e-mail] ---
(In reply to Fabian Maurer from comment #16)
> Ah, so you do need Win10, Win7 doesn't work.

I can't seem to solve the problem using the settings of windows10 in wine, in
figure3

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #19 from [e-mail] ---
Created attachment 73510
  --> https://bugs.winehq.org/attachment.cgi?id=73510
Figure 3 on debian

[Bug 53929] ExtTextOut() does not support surrogate pairs

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53929

--- Comment #20 from Fabian Maurer  ---
I mean, it only works on Windows 10, but not on Windows 7.