[Bug 57097] New: Keyboard inputs for special characters on non-english keyboards are switched in winewayland

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57097

Bug ID: 57097
           Summary: Keyboard inputs for special characters on non-english
                    keyboards are switched in winewayland
           Product: Wine
           Version: 9.16
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: winewayland
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

I noticed that if I use a german keyboard layout, the key mapping seems to be
off. I'm using kwin_wayland (but also tested on sway, same result).

The regular lower keys are ok, but example, ö produces ü, # produces ä, etc.

Looking at dlls/winewayland.drv/wayland_keyboard.c, I saw that there are
different lookup tables for different languages (scan2vk_qwerty,
scan2vk_azerty. scan2vk_qwertz, ...), so my first quess was that something
might be switched there.
But that doesn't really seem the case, changing it to always use
`scan2vk_qwertz ` regardless of `lang` makes it work.
I don't really have a very good understanding of how this is supposed to work,
but does the appliaction really need to do such a translation? Isn't that
already handled by the layout being passed in? At least that would explain why
it works when doing no such lookup at all.

[Bug 57097] Keyboard inputs for special characters on non-english keyboards are switched in winewayland

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57097

Ken Sharp  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |download, localization,
                   |                            |source

[Bug 57097] Keyboard inputs for special characters on non-english keyboards are switched in winewayland

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57097

--- Comment #1 from mata  ---
Created attachment 77568
  --> https://bugs.winehq.org/attachment.cgi?id=77568
ignore switch that breaks keysyms

[Bug 57097] Keyboard inputs for special characters on non-english keyboards are switched in winewayland

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57097

--- Comment #2 from mata  ---
Tested again in wine-10.0rc1, issue is still present.

Skipping the switch that selects the lookup table still works around it.

[Bug 57097] Keyboard inputs for special characters on non-english keyboards are switched in winewayland

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57097

Fell  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #3 from Fell  ---
This is still present in Wine 10.1

Steps to reproduce:

1. Start a Wayland session in KDE Plasma.
2. Set the keyboard layout to something nonstandard, like German QWERTZ.
3. Run `DISPLAY= wine notepad` to run Notepad using the native Wayland driver.
4. Begin typing some of the characters mentioned below.
5. Observe, that some of the produced characters to not correspond to what is
typed.

Some observations:

all of the following are mapped correctly: !"§$%&/()=¹²³{[]}|<>@:.
+-*/ on the numpad are mapped correctly
A-X are all mapped correctly, Y and Z are swapped
All digits (numpad and not numpad) are mapped correctly
typing Ä produces Ö
typing Ö produces Ü
typing Ü produces ? (actual question mark)
typing ° produces *
typing ^ produces +
typing + (not numpad) produces nothing at all (the cursor doesn't advance)
typing * (not numpad) produces `
typing ` or ´ produces nothing at all
typing ? produces _
typing ß procuces -
typing € produces 0x80, which is correct in Windows-1252 encoding, but is
rendered as a vertical bar, slightly shorter than | (this might be a whole
different bug)

System Information:

Wine Version: 10.1 (from official Arch Linux repository)
Operating System: Arch Linux 
KDE Plasma Version: 6.3.0
KDE Frameworks Version: 6.10.0
Qt Version: 6.8.2
Kernel Version: 6.13.2-arch1-1 (64-bit)
Graphics Platform: Wayland
Processors: 24 × AMD Ryzen 9 3900X 12-Core Processor
Memory: 62.7 GiB of RAM
Graphics Processor: NVIDIA GeForce RTX 2080 Ti

[Bug 57097] Keyboard inputs for special characters on non-english keyboards are switched in winewayland

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57097

--- Comment #4 from Fell  ---
This is still present in Wine 10.1

Steps to reproduce:

1. Start a Wayland session in KDE Plasma.
2. Set the keyboard layout to something nonstandard, like German QWERTZ.
3. Run `DISPLAY= wine notepad` to run Notepad using the native Wayland driver.
4. Begin typing some of the characters mentioned below.
5. Observe, that some of the produced characters to not correspond to what is
typed.

Some observations:

all of the following are mapped correctly: !"§$%&/()=¹²³{[]}|<>@:.
+-*/ on the numpad are mapped correctly
A-X are all mapped correctly, Y and Z are swapped
All digits (numpad and not numpad) are mapped correctly
typing Ä produces Ö
typing Ö produces Ü
typing Ü produces ? (actual question mark)
typing ° produces *
typing ^ produces +
typing + (not numpad) produces nothing at all (the cursor doesn't advance)
typing * (not numpad) produces `
typing ` or ´ produces nothing at all
typing ? produces _
typing ß procuces -
typing € produces 0x80, which is correct in Windows-1252 encoding, but is
rendered as a vertical bar, slightly shorter than | (this might be a whole
different bug)

System Information:

Wine Version: 10.1 (from official Arch Linux repository)
Operating System: Arch Linux 
KDE Plasma Version: 6.3.0
KDE Frameworks Version: 6.10.0
Qt Version: 6.8.2
Kernel Version: 6.13.2-arch1-1 (64-bit)
Graphics Platform: Wayland
Processors: 24 × AMD Ryzen 9 3900X 12-Core Processor
Memory: 62.7 GiB of RAM
Graphics Processor: NVIDIA GeForce RTX 2080 Ti

[Bug 57097] Keyboard inputs for special characters on non-english keyboards are switched in winewayland

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57097

--- Comment #5 from Fell  ---
Sorry for the double posting earlier. That was a mistake.

I can now also confirm that the workaround by @mata does indeed still fix the
issue. The code in question seems to handle specifically French, German, Swiss
and various "dvorak" layouts, which seems odd.
```
switch (lang)
{
case MAKELANGID(LANG_FRENCH, SUBLANG_DEFAULT): scan2vk = scan2vk_azerty; break;
case MAKELANGID(LANG_GERMAN, SUBLANG_DEFAULT): scan2vk = scan2vk_qwertz; break;
case MAKELANGID(LANG_GERMAN, SUBLANG_GERMAN_SWISS): scan2vk = scan2vk_qwertz;
break;
default: scan2vk = scan2vk_qwerty; break;
}
if (strstr(xkb_layout, "dvorak")) scan2vk = scan2vk_dvorak;
```
It was added in this commit:
https://gitlab.winehq.org/wine/wine/-/commit/b2d16b920431820121c849e8bbc462c...

Could this have been a workaround for a potentially wrong keyboard handling in
some other Wayland compositor than KWin?

Could someone try this in other desktop environments?

[Bug 57097] Keyboard inputs for special characters on non-english keyboards are switched in winewayland

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57097

Mershl  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 57097] Keyboard inputs for special characters on non-english keyboards are switched in winewayland

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57097

--- Comment #6 from mata  ---
Still no change in wine 10.4

I've tried to make some sense of those remappings, it seems to try to fix the
layout internally, but the xcb layout should already do that.
But even with that assumption it doesn't seem to work, e.g. it doesn't really
translate a qwerty layout to qwertz if the lookup table is selected
accordingly.
And if that was required, it would need to have similar tables for any other
potential layout.

The compositors with which I tried it are kwin_wayland, weston and sway - all
with the same result.

Another minor problem I saw is that the numlock state can't be toggled.
A while ago I remember it being stuck in off state, now it's always on.
Not sure if that's somewhow related.

[Bug 57097] Keyboard inputs for special characters on non-english keyboards are switched in winewayland

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=57097

Rémi Bernon  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #7 from Rémi Bernon  ---
The switch is correct and it is there to set the proper Win32 scan code to vkey
tables depending on the keyboard layout language. The issue with winewayland
layouts seems to be coming from a mismatch in keyboard layout selection:

Whenever winewayland receives keymap change notification, we change the focused
window active layout by sending it an WM_INPUTLANGCHANGEREQUEST window message.
This message, then calls NtUserActivateKeyboardLayout with the selected layout.
This sends a WM_INPUTLANGCHANGE message to the focused window, changing its
thread info kbd_layout to the active layout.

When winewayland receives a keyboard input, we send the keyboard input as a raw
scancode, which is supposed to later be translated to the proper vkey according
to the active keyboard layout.

However, that conversion happens in send_hardware_message which is called
directly from NtUserSendHardwareMessage, which is itself called directly from
the winewayland keyboard input handler, in the context of the winewayland event
thread, which is an internal thread and different from any window thread.

That thread never changed its active layout and still uses the default keyboard
layout, which is always matching the user locale (may be changed with LC_ALL
env var).

The keyboard layout problem can be worked around by setting LC_ALL= to the
language of the desired keyboard layout.

It then could probably be fixed by calling NtUserActivateKeyboardLayout in the
context of the winewayland thread, in addition to sending the
WM_INPUTLANGCHANGEREQUEST message, although that message could also be hooked
by application to refuse such keyboard layout activation and that should be
handled.

Alternatively, the hardware message scan code -> vkey conversion could be moved
to the receiving window thread, which has the right keyboard layout activated,
but that would require some changes in wineserver, as it currently relies on
keyboard hardware message vkey being set.

[Bug 57097] Keyboard inputs for special characters on non-english keyboards are switched in winewayland

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=57097

--- Comment #8 from Mershl  ---
can confirm the proposed LC_ALL=de_DE of Rémi works around the issue. tested on
german keyboardlayout and winewayland 10.5

[Bug 57097] Keyboard inputs for special characters on non-english keyboards are switched in winewayland

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=57097

Bungeefan  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]