[Bug 27406] New: Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

Summary: Red Faction deadlock on start
           Product: Wine
           Version: 1.3.21
          Platform: x86
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: directx-d3d
        AssignedTo: [e-mail]
        ReportedBy: [e-mail]

Red Faction (version 1.2) fails to start, with a message like this:

err:ntdll:RtlpWaitForCriticalSection section 0x7d63d240 "wined3d_main.c:
wined3d_cs" wait timed out in thread 0028, blocked by 0009, retrying (60 sec)

This game is rated Platinum in AppDB, so I checked with Wine 1.1.21, and indeed
it works perfectly with such old Wine version. Bisecting revealed that it broke
between 1.1.34 and 1.1.35, and the patch that broke it is:

0e270bbd226c34bd2831673c1c546f8db76b0487 is the first bad commit
commit 0e270bbd226c34bd2831673c1c546f8db76b0487
Author: Henri Verbeet 
Date:   Mon Dec 14 20:49:52 2009 +0100

wined3d: Send window messages through the swapchain before sending them to
the application.

:040000 040000 82a4d1078cd0ab3156aba80508dbd3d830b37dee
4cd0938c1bf4a1e0f3bb09ba258447f0fe5a98ed M    dlls

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

Piotr Pawlow  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |regression

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

Alex Bradbury  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

--- Comment #1 from Piotr Pawlow  2011-06-06 17:52:04 CDT ---
Created an attachment (id=35049)
 --> (http://bugs.winehq.org/attachment.cgi?id=35049)
d3d+message+tid trace

This is how I understand what happens:

There are 2 threads involved: thread 0020 manages the window, and thread 0009
does the D3D stuff.

Thread 0009:
- calls CreateDevice with hFocusWindow being the window from the other thread
- wined3d hooks its wndproc to the window (not shown in the trace, as
wined3d_register_window function has no TRACE call)
- wined3d_device_create calls wined3d_device_acquire_focus_window with wined3d
mutex held
- wined3d_device_acquire_focus_window calls SetWindowPos
- SetWindowPos sends WM_WINDOWPOSCHANGING message to the other thread

Thread 0020:
- the message gets passed to wined3d_wndproc
- wined3d_wndproc tries to acquire the mutex, which is held by the other
thread, which in turn is waiting for the message to get processed == deadlock.

So, the solution seems to be either:
- never call any API function that may send a message while holding the mutex
or
- somehow get rid of the mutex in the wined3d_wndproc

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

--- Comment #2 from Piotr Pawlow  2011-06-06 18:39:29 CDT ---
Created an attachment (id=35050)
 --> (http://bugs.winehq.org/attachment.cgi?id=35050)
Separate mutex for wndproc table operations

I don't see anything there worth protecting with the main wined3d mutex, it's
just wndproc table operations, so how about using a separate mutex for wndproc
only?

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

Piotr Pawlow  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |patch

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

--- Comment #3 from Henri Verbeet  2011-06-07 05:10:22 CDT ---
(In reply to comment #2)
> Created an attachment (id=35050)

--> (http://bugs.winehq.org/attachment.cgi?id=35050) [details]
> Separate mutex for wndproc table operations
>
> I don't see anything there worth protecting with the main wined3d mutex, it's
> just wndproc table operations, so how about using a separate mutex for wndproc
> only?

Looks ok in principle, but note that device_process_message() also accesses
device->filter_messages and device->focus_window.

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

--- Comment #4 from Piotr Pawlow  2011-06-07 07:01:15 CDT ---
(In reply to comment #3)
> Looks ok in principle, but note that device_process_message() also accesses
> device->filter_messages and device->focus_window.

The device_process_message() call already wasn't protected by wined3d_mutex, so
I think the patch doesn't change anything here.

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

--- Comment #5 from Henri Verbeet  2011-06-07 07:15:56 CDT ---
(In reply to comment #4)
> (In reply to comment #3)
> > Looks ok in principle, but note that device_process_message() also accesses
> > device->filter_messages and device->focus_window.
>
> The device_process_message() call already wasn't protected by wined3d_mutex, so
> I think the patch doesn't change anything here.

You're right, the patch should be ok in that sense then. I do think those
variables need some protection, but it's a separate issue then.

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

--- Comment #6 from Henri Verbeet  2011-06-13 06:47:31 CDT ---
Are you intending to send this patch in?

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

--- Comment #7 from Piotr Pawlow  2011-06-17 11:44:28 CDT ---
If you think the patch is ok then sure, I can send it.

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

--- Comment #8 from Henri Verbeet  2011-06-17 11:47:13 CDT ---
(In reply to comment #7)
> If you think the patch is ok then sure, I can send it.

Yeah, I think that's ok.

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

--- Comment #9 from Piotr Pawlow  2011-06-22 14:58:07 CDT ---
Commited (b5e04640cc358dda13f70961c31295b522611577), and fixed I guess. Can't
test it, as the game crashes at start with current git, but it's a separate
issue.

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

--- Comment #10 from Piotr Pawlow  2011-06-22 15:10:29 CDT ---
Update: it crashes in gameoverlayrenderer, so that's a Steam Community related
problem. It worked before, I don't know whether some Wine changes or Steam
updates broke it. Disabling Steam Community makes the game work, so this bug
can be closed. Thank you.

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

Henri Verbeet  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |FIXED

--- Comment #11 from Henri Verbeet  2011-06-24 07:28:44 CDT ---
Reported fixed.

[Bug 27406] Red Faction deadlock on start

http://bugs.winehq.org/show_bug.cgi?id=27406

Alexandre Julliard  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #12 from Alexandre Julliard  2011-06-24 17:44:03 CDT ---
Closing bugs fixed in 1.3.23.