[Bug 60493] New: winemac.drv: deadlock between macdrv_surface_flush and macdrv_WindowPosChanged (Unreal Engine games hang on splash screen)

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=60493

            Bug ID: 60493
           Summary: winemac.drv: deadlock between macdrv_surface_flush and
                    macdrv_WindowPosChanged (Unreal Engine games hang on
                    splash screen)
           Product: Wine
           Version: 11.18
          Hardware: x86-64
                OS: MacOS
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: winemac.drv
          Assignee: [e-mail]
          Reporter: [e-mail]
  Target Milestone: ---

**Wine version:** 11.18 (Gcenx macOS build); code unchanged on master as of
2026-10-09
**OS:** macOS 15.5, Apple Silicon (Rosetta 2)

**Summary**

Since commit 1a1d1f3f ("winemac.drv: Hide client_view when flushing window
surfaces"),
`macdrv_surface_flush` calls `get_win_data`, which blocks on `win_data_mutex`.
Window
surface flushes can run while win32u holds its user lock, and
`macdrv_WindowPosChanged` takes
the same two locks in the opposite order, so two threads can deadlock.

Seen with Deep Rock Galactic (UE4): its launcher splash is a layered window
updated from one
thread while the main thread creates the game window. The game stays on the
splash forever
(roughly 6 of 6 launches here). Attaching a debugger unsticks it.

**Stacks (macOS `sample`)**

Thread A (splash):
```
NtUserSetLayeredWindowAttributes
  update_window_state
    apply_window_pos                (get_win_ptr: user lock held)
      update_surface_region
        window_surface_set_shape
          window_surface_flush
            macdrv_surface_flush    (surface.c:125)
              get_win_data          -> waits for win_data_mutex
```

Thread B (main thread, creating the game window):
```
NtUserCreateWindowEx
  ...
    macdrv_WindowPosChanged         (holds win_data_mutex)
      set_cocoa_window_properties   (window.c:248)
        NtUserGetWindowLongW        -> waits for the user lock
```

**Possible fix**

Don't block on `win_data_mutex` in the flush path. For example, a
`try_get_win_data` that
uses `pthread_mutex_trylock` (the mutex is recursive, so a thread that already
holds it still
succeeds); if it's busy, hiding `client_view` is left to the next flush. A
patch doing this
(about 25 lines) fixes the hang here: 4 of 4 launches reach the game window.

```diff
--- a/dlls/winemac.drv/surface.c
+++ b/dlls/winemac.drv/surface.c
-    if ((data = get_win_data(window_surface->hwnd)))
+    if ((data = try_get_win_data(window_surface->hwnd)))
```
(full patch: tools/wine-dxmt/winemac-flush-deadlock.patch in
https://github.com/codeWonderland/neutron)

**Patch (for reference):**
https://github.com/codeWonderland/neutron/blob/main/tools/wine-dxmt/winemac-...

(Analysis and testing were done with AI assistance; the measurements are from
my own Mac. Patches are linked for reference.)

-- 
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.