[Bug 56515] New: Calling "wine reg ..." from python's subprocess lead UnicodeDecodeError if locale isn't us_EN.UTF-8.

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=56515

Bug ID: 56515
           Summary: Calling "wine reg ..." from python's subprocess lead
                    UnicodeDecodeError if locale isn't us_EN.UTF-8.
           Product: Wine
           Version: 9.5
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: -unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

Created attachment 76271
  --> https://bugs.winehq.org/attachment.cgi?id=76271
Python script as an example.

Before concluding this I tested running other programs using Python's
subprocess tools, "rm --help", "ls --help", "man --help", etc. All of them have
it's output displayed normally in the language set by locale LANG environment
variable, every non-English character gets decoded in utf-8 normally.

But here it's what happens when trying run wine reg while scripting in python:

Traceback (most recent call last):
  File "call_wine_reg_script.py", line 31, in <module>
    out, err = res.communicate()
               ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1196, in communicate
    stdout = self.stdout.read()
             ^^^^^^^^^^^^^^^^^^
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc6 in position 6: invalid
continuation byte

This happens because my system locale is set to use "pt_BR.UTF-8" and Wine's
output is trying to output the word "Não" the "ã" gets interpreted as 0xc6 "Æ"
latin-1 as utf-8 which end up in a UnicodeDecodeError.

I attached the script (anyone testing will have to change the prefix path). I
tried setting locale on the script, tried passing the environment variables to
the Popen command, but the outcome is always the same.

In the example script I used ja_JP.UTF-8, but any language that has non-English
character will have the same problem.

I tested with other Wine's commands like wineboot, winecfg, etc, but all of
them displays English text so no problem there, if there's any other Wine's
command that change it's output language based on the LANG environment
variable, I think it will trigger it.

[Bug 56515] Calling "wine reg ..." from python's subprocess lead UnicodeDecodeError if locale isn't us_EN.UTF-8.

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=56515

Eric Pouech  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #1 from Eric Pouech  ---
as of today, reg.exe (like a couple of other wine programs):
- outputs in wide string, converted to unix locale when output is bound to a
console,
- outputs in wide string, converted to windows oem code page (which depends on
your unix locale) when output is bound to a file, pipe...

IIRC it's how Windows behaves (I).

So in your case, you can either:
- force LC_ALL=C for Wine (but output will be in English),
- get the oem cp (GetOemCP() API) for the settings you pass in LC_ALL, and use
that to decode the pipe

[Bug 56515] Calling "wine reg ..." from python's subprocess lead UnicodeDecodeError if locale isn't us_EN.UTF-8.

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=56515

Ken Sharp  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |download, localization,
                   |                            |source, testcase