[Bug 48462] New: "Listen to RawKey* events in the desktop thread" patch makes keyboard unreliable in EverQuest
https://bugs.winehq.org/show_bug.cgi?id=48462
Bug ID: 48462 Summary: "Listen to RawKey* events in the desktop thread" patch makes keyboard unreliable in EverQuest Product: Wine-staging Version: 5.0-rc5 Hardware: x86 OS: Linux Status: UNCONFIRMED Severity: normal Priority: P2 Component: -unknown Assignee: wine-bugs@winehq.org Reporter: dvufhr.nznqarjf@noclue.notk.org CC: leslie_alistair@hotmail.com, z.figura12@gmail.com Regression SHA1: 8218a789558bf074bd26a9adf3bbf05bdb9cb88e Distribution: Fedora
Upgrading from 4.21 to 5.0-rc3 on fedora caused unreliability issues for keyboard in EverQuest (EQ below): - windowed EQ window sometimes (!) taking input when not focused, even when hidden in another workspace - modifier keys (shift, alt, ctrl) sometimes working, sometimes not.. For example right shift seemed to work well exactly once immediately after defocusing/refocusing the window, but not after that, while the left shift key worked a bit better for some reason. I couldn't figure out why or what he hell is happening.
Well, anyway, game input goes pretty bonkers and the game is much more difficult to play. I know EQ makes heavy use of raw inputs itself, it reprocesses all keys not as input letters but as key presses and this leads to some weird input sometimes (e.g. typing fast enough with a bit of graphical lag will queue input letters, but pressing shift then will affect letters currently being processed in the backlog, this is pretty interesting and probably not wine's fault); but this patch takes it to a whole new level.
I bisected the staging tree down to this patch: patches/user32-rawinput-keyboard/0004-winex11.drv-Listen-to-RawKey-events-in-the-desktop-t.patch introduced in staging commit 8218a789 ("Updated rawinput patchsets")
Happy to test things if required, or report somewhere else if that helps.
Thanks!
https://bugs.winehq.org/show_bug.cgi?id=48462
Dominique Martinet dvufhr.nznqarjf@noclue.notk.org changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |rbernon@codeweavers.com
https://bugs.winehq.org/show_bug.cgi?id=48462
Dominique Martinet dvufhr.nznqarjf@noclue.notk.org changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |dvufhr.nznqarjf@noclue.notk | |.org
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #1 from Rémi Bernon rbernon@codeweavers.com --- Are you using Wine in virtual desktop mode?
It could be a duplicate of https://bugs.winehq.org/show_bug.cgi?id=48419
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #2 from Dominique Martinet dvufhr.nznqarjf@noclue.notk.org --- No, virtual desktop is disabled; this is just a floating window with the game's built-in windowed mode.
The result does look similar though, whatever other bug report I was doing at the time got sent on in game chat and people wondered what the hell I was saying :D
I would guess desktop mode also interprets keypresses through raw mode (because it would need to send raw events to game which expect them), so your patch make keystrokes go to it when unfocused; and my game has the same problem. wine desktop mode might be "work-around-able" (e.g. only forward keys if focused) but I don't see how closed source games could be :/
Thanks,
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #3 from Rémi Bernon rbernon@codeweavers.com --- AFAICS this is an issue with how focus is handled in Wine.
Sometimes, for instance right after startup, the game tries to bring itself to foreground by calling SetForegroundWindow. This succeeds on the Win32 side of Wine, and the rest of the processing works as if the game window has foreground: it can acquire dinput devices, raw input events are dispatched to it, etc...
However this is not always reflected on the X11 side, where another window may have the input focus.
Before the rawinput patches, the input events were received on each individual windows, and not really having focus would mean that no input event would be received regardless of what the Win32 side would expect. Listening to XI2 raw events on the root window, now makes Wine always receive the input events and this focus inconsistency becomes much more problematic.
I think using virtual desktop would be a possible workaround once the other issue with this mode is fixed, but it's not a real solution.
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #4 from Rémi Bernon rbernon@codeweavers.com --- Another thing that could cause trouble is that the game sometimes doesn't process its messages for a long time; it's the case during startup. So react to X11 focus events to enable or disable rawinput events may not be enough anyway, as the focus events would only be handled when the game checks for Win32 messages and rawinput subscription could stay enabled for a while.
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #5 from Dominique Martinet dvufhr.nznqarjf@noclue.notk.org ---
However this is not always reflected on the X11 side, where another window may have the input focus.
And it's not possible to have a better synchro between the windows state and the x11 state ? FWIW, in wayland, you cannot get events for the root window anymore (only the really focused wayland window gets events) -- that looks much saner to me.
virtual desktop as a workaround
That is a good point, I might get better input with that for my other problems too, I'll give it a spin. But I don't think it's enough to just fix virtual desktop, you will probably have many other games behaving the same way I think?
no events processed for a long time
Now while that might be very true for games like EQ (during heavy lag period I can easily have something like 10s delay between what I finished typing and the moment it's done processing), I don't think it was the case when I had problems with what I typed in firefox being sent to the game as well. In the occurrences I had the game was idle in a relatively lag-free safe area so there shouldn't have been even a second of lag there, and text I was typing minutes after switching workspace was still sent.
There might be a problem with how wmii notifies x11 clients of workspace switch (just tested with xev: I see proper FocusIn/FocusOut events if focused and UnmapNotify/MapNotify events even if not focused so it looks proper to me)... But anyway, I still don't find natural that games have to process root window key events. Call me paranoid but I don't particularily trust windows programs I haven't seen the source of, so the less it can see of what I don't type with it explicitely focused the better (think keyloggers or similar)
Thanks,
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #6 from Rémi Bernon rbernon@codeweavers.com ---
And it's not possible to have a better synchro between the windows state and the x11 state ?
It's possible to force the Windows state to match the X11 but in this case Wine may not behave as close to Windows anymore, and that could cause some other issues.
FWIW, in wayland, you cannot get events for the root window anymore (only the really focused wayland window gets events) -- that looks much saner to me.
It depends what you want to do. Wine being an implementation of the whole system, I think it cannot work perfectly with only the knowledge of a few windows in the middle of an unknown environment.
That is a good point, I might get better input with that for my other problems too, I'll give it a spin. But I don't think it's enough to just fix virtual desktop, you will probably have many other games behaving the same way I think?
I haven't seen many games trying to forcefully get focus by calling SetForegroundWindow on their window, but possibly there are others. Also the keyboard patch which is causing the problem is the least tested patch of the series, for the reason you describe below.
Now while that might be very true for games like EQ (during heavy lag period I can easily have something like 10s delay between what I finished typing and the moment it's done processing), I don't think it was the case when I had problems with what I typed in firefox being sent to the game as well. In the occurrences I had the game was idle in a relatively lag-free safe area so there shouldn't have been even a second of lag there, and text I was typing minutes after switching workspace was still sent.
Sure I was only mentioning that because I encountered the problem while trying to write a fix: subscribing to the events on FocusIn and unsubscribing on FocusOut may not be enough because the FocusOut event may be processed late.
But anyway, I still don't find natural that games have to process root window key events. Call me paranoid but I don't particularily trust windows programs I haven't seen the source of, so the less it can see of what I don't type with it explicitely focused the better (think keyloggers or similar)
This is a perfectly valid concern although one could argue that keylogguers are genuine Windows application. It's also the reason why the keyboard patch is kept separate -and maybe should be disabled by default.
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #7 from Rémi Bernon rbernon@codeweavers.com --- Created attachment 66252 --> https://bugs.winehq.org/attachment.cgi?id=66252 Patch for staging
Is this fixing anything? -more likely in staging because of the _NET_ACTIVE_WINDOW patch-
It should at least fix the virtual desktop mode by not listening to raw input events anymore when not focused.
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #8 from Dominique Martinet dvufhr.nznqarjf@noclue.notk.org --- Just tested, wine staging on 5.0-rc5 up until "winex11.drv: Listen to RawKey* events in the desktop thread.", plus the attached 3 commits, plus "winex11.drv: Add support for _NET_ACTIVE_WINDOW. (v2)" to make it compile:
- no virtual desktop, EQ, same behaviour - virtual desktop & EQ, same behaviour
I actually tried virtual desktop and something that doesn't exhibit the behaviour (e.g. notepad) and I do not reproduce the problem with virtual desktop BUT the point in time I'm testing doesn't have winex11-key_translation mentioned in the other bug; I'll try again with the full staging stack + the three attached commits just in case that changes anything.
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #9 from Rémi Bernon rbernon@codeweavers.com --- It may also depend on some window manager features, I don't know how widespread EWMH implementation is, and these patch depend on _NET_ACTIVE_WINDOW property updates. To see if anything is happening on focus change, WINEDEBUG=+event and grep "X11DRV_PropertyNotify" could be used. When not using virtual desktop there should be messages like: - Setting foreground to %p/%lx to match X11 state. - Active window is not Wine's, setting foreground to desktop. - No active window, setting foreground to desktop.
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #10 from Dominique Martinet dvufhr.nznqarjf@noclue.notk.org --- Created attachment 66255 --> https://bugs.winehq.org/attachment.cgi?id=66255 EQ log with WINEDEBUG=+event
It may also depend on some window manager features, I don't know how widespread EWMH implementation is, and these patch depend on _NET_ACTIVE_WINDOW property updates.
Right, using wmii here might be something very different.
So to finish my recap, on top of full staging 5.0-rc5 I do not notice any difference with your three last patches either: - EQ without virtual desktop behaves exactly the same - with virtual desktop and EQ, there is a difference between the commit I was testing and staging, but no difference with your extra three patches. Basically, directly losing focus used to stop input until I changed workspace and came back (unmap/map), which then took input again even if not focused. On staging, directly losing focus no longer stops input at all. I'll try to translate all of these actions to winedebug messages, didn't know about these it might be better than xev for you... - still cannot reproduce the "bad" input with virtual destkop and notepad, so might be game-dependant for the other bz
Either way, the last three patches do not make a difference for my problem here, be it on top of the "Listen to RawKey* events" commit or all staging patches as of 5.0-rc5
WINEDEBUG=+event and grep "X11DRV_PropertyNotify" could be used.
Tested EQ, no virtual desktop, on top of all staging patches + your 3 latest patches. I've attached a full log (with WINEDEBUG=+event), no grep, added timestamps, and the following schedule ± a couple of seconds: 10s: EQ mostly ready, anything before that can be ignored (window starting, a few keypresses to get into an input field) 15s: 3 key presses: these work as expected, game is focused. 20s: focus out 25s: 3 key presses: these don't appear in window. 30s: change workspace out 35s: 3 key presses: still don't appear 40s: change workspace in 45s: 3 key presses: these are taken by game but x11 window isn't focused 50s: change workspace out again 55s: 3 key presses: these do appear (can notice when going back in) 60s: change workspace back in and ^C after confirming there are 9 characters in field as "expected" with bug
so if anything the problem is around the 40s mark, the X11DRV_Expose perhaps?
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #11 from Rémi Bernon rbernon@codeweavers.com --- I don't know about wmii, the logs do not show the expected messages so it may not support the required extension. It's possible to be sure of that by using "xev -root -event property" and checking if there's any _NET_ACTIVE_WINDOW property change when switching between applications.
If that's not the case then the patches won't work at all, even with virtual desktop mode. There's another way to implement the fix for the virtual desktop mode, which I'm going to implement as a quick workaround.
For the other mode it's going to be much harder, as the proper fix will require to reconcile Wine and X11 focus state together. Addressing WMs that do not support this extension will only make things worse.
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #12 from Dominique Martinet dvufhr.nznqarjf@noclue.notk.org ---
I don't know about wmii, the logs do not show the expected messages so it may not support the required extension. It's possible to be sure of that by using "xev -root -event property" and checking if there's any _NET_ACTIVE_WINDOW property change when switching between applications.
Neither`xev -root -event property` nor `xev -event property` show anything about _NET_ACTIVE_WINDOW when changing focus ; that would explain why the new patches do not change anything. I guess I could consider implementing that for wmii eventually, but I wouldn't be surprised quite a few other WMs don't implement that -- even just considering "mainstream" window managers I can easily count half a dozen if not more, it'd be a wonder if all of gnome3/x11, gnome3/wayland+xwayland, mate, cinnamon, kde/x11, kde/wayland+xwayland, xfce etc all do it...
Anyway, waiting for alternative desktop mode fix and will test that as well, the help is appreciated. Thanks!
https://bugs.winehq.org/show_bug.cgi?id=48462
Rémi Bernon rbernon@codeweavers.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Attachment #66252|0 |1 is obsolete| |
--- Comment #13 from Rémi Bernon rbernon@codeweavers.com --- Created attachment 66261 --> https://bugs.winehq.org/attachment.cgi?id=66261 Virtual desktop patch for staging, without _NET_ACTIVE_WINDOW
This should fix the virtual desktop mode, without requiring _NET_ACTIVE_WINDOW property updates.
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #14 from Dominique Martinet dvufhr.nznqarjf@noclue.notk.org ---
This should fix the virtual desktop mode, without requiring _NET_ACTIVE_WINDOW property updates.
I can confirm this last patch works on my system - EQ in virtual desktop no longer exhibit the problem.
It has a few limitations (like multiple instances of the game sharing the same virtual desktop, which isn't as comfortable as native x11 windows) but it's a good start; I can live with that for a while.
I'm a bit worried this is really application-specific though, I'll have a look at the code over the weekend now I should have a bit more time for this.
Thanks,
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #15 from Alistair Leslie-Hughes leslie_alistair@hotmail.com --- (In reply to Rémi Bernon from comment #13)
Created attachment 66261 [details] Virtual desktop patch for staging, without _NET_ACTIVE_WINDOW
This should fix the virtual desktop mode, without requiring _NET_ACTIVE_WINDOW property updates.
What patchset did you want me to add this patch to?
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #16 from Rémi Bernon rbernon@codeweavers.com --- I think it should be added to the user32-rawinput-keyboard patchset, as it is the main issue.
https://bugs.winehq.org/show_bug.cgi?id=48462
Dominique Martinet dvufhr.nznqarjf@noclue.notk.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|UNCONFIRMED |RESOLVED
--- Comment #17 from Dominique Martinet dvufhr.nznqarjf@noclue.notk.org --- The patch here only addressed virtual desktop, but that seems also fixed without virtual desktop in 5.1 staging (at least the behaviour I described with unmap/map restoring keyboard focus wrongly no longer happens on whateved fedora ships)
Marking the bug resolved, thanks!
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #18 from Rafał Mużyło galtgendo@o2.pl --- I'm led to believe that comment 17 is based on a mistake: rawinput patchset has been disabled in staging, so the test results might no longer be relevant.
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #19 from Alistair Leslie-Hughes leslie_alistair@hotmail.com --- (In reply to Rafał Mużyło from comment #18)
I'm led to believe that comment 17 is based on a mistake: rawinput patchset has been disabled in staging, so the test results might no longer be relevant.
Fedora ships wine-staging by default, so having this patchset disabled has fixed the issue at hand.
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #20 from Dominique Martinet dvufhr.nznqarjf@noclue.notk.org --- Indeed, if the rawinput patchset has been disabled in staging then the issue would also have gone away; I didn't go as far as to check why the issue disappeared.
I can recompile with the patchs to check again, if that's any help, but unlike what I had said previously it would be hard to find proper time to debug so not sure how much helpful it is. I'll at least reopen here if the issue manifests itself again when patches get reapplied if I'm still using the game at that point :)
https://bugs.winehq.org/show_bug.cgi?id=48462
--- Comment #21 from Rafał Mużyło galtgendo@o2.pl --- The thing is this bug was one of the reasons the patchset was disabled and without that patchset there's - for example - no keyboard input in some other games.
I'm not sure if the patchset still works though, as I don't use full staging and user32-rawinput-keyboard portion depends on quite a few other series...
Damn it.
https://bugs.winehq.org/show_bug.cgi?id=48462
Zebediah Figura z.figura12@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Resolution|FIXED |DUPLICATE
--- Comment #22 from Zebediah Figura z.figura12@gmail.com --- I think this is a duplicate of bug 48419.
*** This bug has been marked as a duplicate of bug 48419 ***
https://bugs.winehq.org/show_bug.cgi?id=48462
Austin English austinenglish@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |CLOSED
--- Comment #23 from Austin English austinenglish@gmail.com --- Closing.
-
WineHQ Bugzilla