[Bug 27620] New: RIFT 1.3 hotfix 2 hangs, weird behavior with vectored exception handlers

http://bugs.winehq.org/show_bug.cgi?id=27620

Summary: RIFT 1.3 hotfix 2 hangs, weird behavior with vectored
                    exception handlers
           Product: Wine
           Version: unspecified
          Platform: x86
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: ntdll
        AssignedTo: [e-mail]
        ReportedBy: [e-mail]

Created an attachment (id=35305)
 --> (http://bugs.winehq.org/attachment.cgi?id=35305)
Trace with unchanged wine 1.3.23, +process,+seh,+tid

The game RIFT hangs as of 1.3 hotfix 2; after logging in, the game window
appears and the startup music plays, but does not display the progress display
for loading characters.

This is being reported widely on a number of configurations, but I'm on Ubuntu
11.04, NVidia GeForce GT 430 w/ 270.41.06 drivers as shipped by Ubuntu.  Tested
on everything from the C-Korn PulseAudio-patched PPA Wine (1.3.22) to current
git HEAD (tagged as wine-1.3.23).

I've been trying to figure this out on my own, and have made some progress. 
The problem is in the vectored exception handlers; one of the threads tries to
set a name for itself via the 406d1388 exception trick.  Unfortunately, the top
vectored exception handler list entry appears to be garbage, and its handler
function almost immediately throws an c0000005 exception.  This re-calls the
garbage handler, which trips, causing another call, and so on.  Eventually the
thread runs out of stack and is forcibly aborted.  Since the other threads are
waiting on this thread to do something interesting, the game hangs.

I've patched ntdll to include some extra tracing information and debug hooks,
including a complete dump of the vectored handler list on every exception and
every call to RtlAddVectoredExceptionHandler.  In this trace, all calls to
RtlAddVectoredExceptionHandler appear to make sense; the call is made, and the
resulting list reflects the current call and all known preceding calls.  But
then, when the exception to add the thread name is hit, a new exception handler
appears to have been added before the others without calling
RtlAddVectoredExceptionHandler.

I was able to set a watch on the list, with a condition that basically meant
"don't trip if we're in RtlAddVectoredExceptionHandler".  As far as I can tell,
we're inside the RIFT executable at that point, so I'm at the end of my
abilities here.

Will attach traces with unpatched and patched Wine, plus a debug transcript for
setting the watchpoint.