[Bug 52936] New: winhttp:url assumes 0xfb00 cannot be converted to the ANSI codepage, fails with UTF-8 codepage

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52936

Bug ID: 52936
           Summary: winhttp:url assumes 0xfb00 cannot be converted to the
                    ANSI codepage, fails with UTF-8 codepage
           Product: Wine
           Version: unspecified
          Hardware: x86-64
                OS: Windows
            Status: NEW
          Severity: normal
          Priority: P2
         Component: winhttp
          Assignee: [e-mail]
          Reporter: [e-mail]

winhttp:url assumes 0xfb00 cannot be converted to the ANSI codepage and thus
fails fails on Windows with the UTF-8 codepage:

url.c:306: Test failed: expected failure
url.c:307: Test failed: got 0
url.c:346: Test failed: expected failure
url.c:347: Test failed: got 0

https://test.winehq.org/data/patterns.html#winhttp:url

These two pairs of failures happen when calling WinHttpCreateUrl() on the
following URLs respectively:

39 static WCHAR escape3[]  = {'?','t','e','x','t','=',0xfb00,0};
  40 static WCHAR escape4[]  = {'/','t','e','x','t','=',0xfb00,0};

So maybe these tests should be skipped when running in a UTF-8 locale since
it's supposed to be able to handle any UTF-16 codepoint.

What is surprising is that this test does not fail on debian11-hi-IN (where
GetACP()==UTF-8). That looks like a bug too.

The tests were introduced in this commit:

commit 7e5b5cb1aa253ce2f20a790989d0f18bcbcab946
Author: Hans Leidekker 
Date:   Tue Nov 6 16:09:04 2018 +0100

winhttp: Properly handle request paths with Unicode characters.

Wine-Bug: https://bugs.winehq.org/show_bug.cgi?id=46076
    Signed-off-by: Hans Leidekker 
    Signed-off-by: Alexandre Julliard

[Bug 52936] winhttp:url assumes 0xfb00 cannot be converted to the ANSI codepage, fails with UTF-8 codepage

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52936

François Gouget  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |source, testcase

[Bug 52936] winhttp:url assumes 0xfb00 cannot be converted to the ANSI codepage, fails with UTF-8 codepage

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52936

--- Comment #1 from Hans Leidekker  ---
(In reply to François Gouget from comment #0)
> winhttp:url assumes 0xfb00 cannot be converted to the ANSI codepage and thus
> fails fails on Windows with the UTF-8 codepage:
>
> url.c:306: Test failed: expected failure
> url.c:307: Test failed: got 0
> url.c:346: Test failed: expected failure
> url.c:347: Test failed: got 0
>
> https://test.winehq.org/data/patterns.html#winhttp:url
>
> These two pairs of failures happen when calling WinHttpCreateUrl() on the
> following URLs respectively:
>
> 39 static WCHAR escape3[]  = {'?','t','e','x','t','=',0xfb00,0};
>   40 static WCHAR escape4[]  = {'/','t','e','x','t','=',0xfb00,0};
>
> So maybe these tests should be skipped when running in a UTF-8 locale since
> it's supposed to be able to handle any UTF-16 codepoint.

Is Microsoft planning to switch the codepage default to UTF-8 across the board?

The new behavior makes sense, but I think I'd rather wait until this feature
comes out of beta, fix Wine and then mark the old behavior as broken.

[Bug 52936] winhttp:url assumes 0xfb00 cannot be converted to the ANSI codepage, fails with UTF-8 codepage

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52936

--- Comment #2 from François Gouget  ---
I don't know what Microsoft's plans are.

What I can tell is that:
* This feature has been present since at least Windows 10 1809v2 but is missing
in Windows 1709.
* What seems like the old way of setting the locales (using intl.cpl) provides
no way of setting a UTF-8 codepage and will fail when trying to set the system
locale to a Unicode-only locale.
* But the PowerShell API will automatically set a UTF-8 codepage when trying to
set the system locale to a Unicode-only locale.
* Seeing some of the bugs I suspect the corporations in India and other places
are probably being careful not to enable UTF-8 yet.

Can we run Wine in the Hindi locale without enabling the UTF-8 codepage? (which
presumably means setting the system locale to some English codepage)

[Bug 52936] winhttp:url assumes 0xfb00 cannot be converted to the ANSI codepage, fails with UTF-8 codepage

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52936

Hans Leidekker  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED
      Fixed by SHA1|                            |8e942f9e413b1bfbe70a1269e7c
                   |                            |02fa534fe61ff

--- Comment #3 from Hans Leidekker  ---
Fixed with 8e942f9e413b1bfbe70a1269e7c02fa534fe61ff.

[Bug 52936] winhttp:url assumes 0xfb00 cannot be converted to the ANSI codepage, fails with UTF-8 codepage

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=52936

Alexandre Julliard  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #4 from Alexandre Julliard  ---
Closing bugs fixed in 8.1.