[Bug 48419] New: user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

Bug ID: 48419
           Summary: user32-rawinput-keyboard: can't use other programs
                    normally while a DirectX11 game is running (virtual
                    desktop mode)
           Product: Wine-staging
           Version: 5.0-rc3
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: -unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
                CC: [e-mail], [e-mail]
      Distribution: ---

When user32-rawinput-keyboard is being used, it's problematic to send keys to
other X11 windows while a game is running.

In detail:
1. configure "virtual desktop" mode via winecfg
2. Start a game in Wine
3. alt-tab to another X11 window (e.g. Firefox)
4. type something (e.g. enter a url in Firefox)
When typing in Firefox, the keys also get send into the game. E.g. if "g" is
being pressed in Firefox, the action which is configured for "g" in the game
will also be triggered.

So when looking up something in Firefox, it will probably trigger some stupid
actions in the game and your spaceship will explode or whatever...

Workaround:
Use wine-staging without applying user32-rawinput-keyboard and without
winex11-key_translation (which is depending on user32-rawinput-keyboard).
./patches/patchinstall.sh DESTDIR=../wine-5.0-rc4 --all -W
user32-rawinput-keyboard -W winex11-key_translation

Tested Wine version: staging-5.0-rc4
Didn't test it, but this should also apply to wine-staging-5.0-rc3.

Tested game: Elite Dangerous (Fullscreen mode) on DXVK-1.5
(probably only affecting DirectX11 applications)

I know, this isn't a lot of technical information.
If the problem isn't immediately clear, don't hesitate to ask me for more
details!

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

--- Comment #1 from Alistair Leslie-Hughes  ---
Can you please attached a +rawinput log?

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

--- Comment #2 from kolAflash  ---
Created attachment 66193
  --> https://bugs.winehq.org/attachment.cgi?id=66193
+rawinput log (running Elite Dangerous)

(In reply to Alistair Leslie-Hughes from comment #1)
> Can you please attached a +rawinput log?

Started with:
env WINEDEBUG=+rawinput WINEARCH=win64
WINEPREFIX="${HOME}"/wines/elite_dangerous/
~/opt/wine-versions/wine-staging-5.0-rc4/bin/wine explorer.exe
/desktop=shell,1600x900 "C:\Program Files
(x86)\Frontier\EDLaunch\EDLaunch.exe"

One second before the game went fullscreen and captured the mouse (winecfg
mouse capturing enabled), I clicked one the address bar of Firefox, beside the
Wine virtual desktop window. So the Firefox window is in focus. I can see that
clearly, by the colour my window manager (kwin_x11) gives to the titlebar of
the Firefox window.

After the game started to it's main menu, I typed "https://www.winehq.org" into
the Firefox address bar, chose an entry from browser history via arrow-up and
-down, and pressed "enter".
Ingame, those keys caused the "exit" button to be marked, so "enter" closed the
game.

Finally I switched to the Wine virtual desktop window via ALT-TAB, pressed
ALT-F3 to open the window manager menu for the Wine window and chose "close",
so Wine was triggered to shut down.

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

--- Comment #3 from kolAflash  ---
P.S.
Actually it's the same for mouse click events. But that's not as annoying as
for the keyboard. So for me it's more a "nice to have" to fix that mouse stuff.

Example:
Clicking onto another window, while the mouse cursor is ingame over a button,
will trigger that button ingame.

Disabling user32-rawinput-keyboard won't change this behaviour. But I guess
it's probably one of the other user32-rawinput-* patchsets.
(I didn't had time to test that yet, but it maybe user32-rawinput-hid,
user32-rawinput-mouse or user32-rawinput-mouse-experimental)

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

Alistair Leslie-Hughes  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #4 from Alistair Leslie-Hughes  ---
Without writing some tests, I have a feeling that the server isn't sending out
"wparam = RIM_INPUTSINK;", when the native applications has focus.

Have you tested this scenario on windows?

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

--- Comment #5 from kolAflash  ---
(In reply to Alistair Leslie-Hughes from comment #4)
> Without writing some tests, I have a feeling that the server isn't sending
> out "wparam = RIM_INPUTSINK;", when the native applications has focus.

Is there a ways I can test that hypothesis?
> Have you tested this scenario on windows?

I don't have a Windows to test :-/

But I tested "Heroes of the Storm" (HotS) on the same machine. And although
HotS is also a DirectX11 game, the bug didn't show up (tested with and without
DXVK). Neither the keyboard, nor the mouse problem was reproducible with HotS.
HotS: https://appdb.winehq.org/objectManager.php?sClass=version&iId=32232
So it looks like towards my first thought, this may only be an Elite Dangerous
bug.
Elite Dangerous:
https://appdb.winehq.org/objectManager.php?sClass=version&iId=37490

Additionally I also disabled user32-rawinput-mouse-experimental and
user32-rawinput-mouse (and also user32-rawinput-nolegacy, user32-rawinput-hid
which depend on the other).
And indeed, the mouse problem in Elite Dangerous also disappeared.
./patches/patchinstall.sh DESTDIR=../wine-5.0-rc4 --all -W
user32-rawinput-keyboard -W winex11-key_translation -W
user32-rawinput-mouse-experimental -W user32-rawinput-mouse -W
user32-rawinput-nolegacy -W user32-rawinput-hid

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

--- Comment #6 from Rémi Bernon  ---
Could you confirm that this is specific to the virtual desktop mode? And also,
whether it happens or not when using virtual desktop, if another application is
focused (winecfg for example) in the desktop window and not the game.

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

--- Comment #7 from kolAflash  ---
(In reply to Rémi Bernon from comment #6)
> Could you confirm that this is specific to the virtual desktop mode? And
> also, whether it happens or not when using virtual desktop, if another
> application is focused (winecfg for example) in the desktop window and not
> the game.

Confirmed!
Bug only appears in virtual desktop mode. And only if Elite Dangerous is in
focus inside the virtual desktop.

Without virtual desktop mode:
If I start Elite Dangerous without wine virtual desktop, the bug doesn't
appear.
If the game runs in fullscreen mode, it gets minimized as soon as another X11
window has focus. And if the game runs in window or borderless window mode, the
bug doesn't happen either (although the game isn't minimized).

In virtual desktop mode:
If I start (for example) winecfg and the winecfg window is in focus, the Elite
Dangerous fullscreen gets minimized and the bug doesn't happen.
If I run Elite Dangerous in windowed mode, the bug appears as long as the Elite
Dangerous window is in focus inside the virtual desktop.

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

--- Comment #8 from Rémi Bernon  ---
Created attachment 66253
  --> https://bugs.winehq.org/attachment.cgi?id=66253
Patch for staging

This should fix the problem in virtual desktop mode by not listening to raw
input events anymore when it is not focused.

It creates a layer of isolation when using virtual desktop mode, but it also
means that in this mode wine can lose some raw events and have its
keyboard/mouse button state get out of sync.

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

Rémi Bernon  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #66253|0                           |1
        is obsolete|                            |

--- Comment #9 from Rémi Bernon  ---
Created attachment 66260
  --> https://bugs.winehq.org/attachment.cgi?id=66260
Virtual desktop patch for staging, without _NET_ACTIVE_WINDOW

This new patch does not require the _NET_ACTIVE_WINDOW property updates to
work. It should fix the virtual desktop mode by not listening to raw events if
it isn't focused.

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

--- Comment #10 from kolAflash  ---
(In reply to Rémi Bernon from comment #9)
> Created attachment 66260 [details]
> Virtual desktop patch for staging, without _NET_ACTIVE_WINDOW
>
> This new patch does not require the _NET_ACTIVE_WINDOW property updates to
> work. It should fix the virtual desktop mode by not listening to raw events
> if it isn't focused.

Works like a charm!

Just tested with wine-staging-5.0.

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

--- Comment #11 from kolAflash  ---
(In reply to kolAflash from comment #10)
> (In reply to Rémi Bernon from comment #9)
> > Created attachment 66260 [details]
> > Virtual desktop patch for staging, without _NET_ACTIVE_WINDOW
> >
> > This new patch does not require the _NET_ACTIVE_WINDOW property updates to
> > work. It should fix the virtual desktop mode by not listening to raw events
> > if it isn't focused.
>
> Works like a charm!

Looks like this patch hasn't made it into wine-staging 5.1 or 5.2.

Is there any problem left to be solved before this can be merged?

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

Gijs Vermeulen  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #12 from Gijs Vermeulen  ---
This patchset was disabled in
https://github.com/wine-staging/wine-staging/commit/d8496cacd170347bbde755ea...

Marking FIXED.

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

Zebediah Figura  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]
                   |                            |.org

--- Comment #13 from Zebediah Figura  ---
*** Bug 48462 has been marked as a duplicate of this bug. ***

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

Zebediah Figura  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED
      Fixed by SHA1|                            |d8496cacd170347bbde755ead06
                   |                            |6be8394fbb82b

--- Comment #14 from Zebediah Figura  ---
Closing bugs fixed in wine-staging 5.0-rc6.

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

--- Comment #15 from kolAflash  ---
Looks good.

I created a separate bug ticket for the mouse issue.
https://bugs.winehq.org/show_bug.cgi?id=49667

[Bug 48419] user32-rawinput-keyboard: can't use other programs normally while a DirectX11 game is running (virtual desktop mode)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48419

Федор Зуев  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]