[Bug 60417] New: ntsync: a mutex whose handles are closed is never abandoned when its owner thread dies (kernel32:sync test_mutex fails with /dev/ntsync)

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=60417

            Bug ID: 60417
           Summary: ntsync: a mutex whose handles are closed is never
                    abandoned when its owner thread dies (kernel32:sync
                    test_mutex fails with /dev/ntsync)
           Product: Wine
           Version: 11.0
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: wineserver
          Assignee: [e-mail]
          Reporter: [e-mail]
  Target Milestone: ---
      Distribution: ---

With in-process synchronization on /dev/ntsync, a thread that dies while owning
a mutex does not abandon it if the process has already closed its handles to
the mutex, even though another thread is still waiting on it. The waiter blocks
forever instead of returning WAIT_ABANDONED. Windows returns WAIT_ABANDONED,
and so does Wine's server-side implementation (without /dev/ntsync).

Wine's own test covers this. The end of test_mutex() in
dlls/kernel32/tests/sync.c creates a mutex, has one thread acquire it and
block, starts a second thread waiting on the mutex, closes the mutex handle,
terminates the owner with TerminateThread(), and expects the waiter to report
WAIT_ABANDONED within a second. On a kernel with /dev/ntsync (here 7.2.7),
kernel32:sync fails the two checks that depend on that:

  sync.c:416: Test failed: got 0x102
  sync.c:420: Test failed: got 0x102

(line numbers in current master as of 2026-09-30; 393 and 397 in Proton
Experimental's tree: the wait for the waiter to signal, and the wait for the
waiter thread to exit). Without /dev/ntsync the test passes. The same test
binary passes on Windows 11 (25H2, build 26200.9457), and a timed version of
the sequence shows the waiter returning WAIT_ABANDONED there 0.1 to 0.7 ms
after TerminateThread() (5 of 5 runs); on ntsync it has not returned after 5 s.
Keeping the mutex handle open until after TerminateThread() makes the ntsync
case work too, so the failure is confined to closed handles. A 32-bit build of
the same timed sequence behaves the same on a WoW64 setup with 32-bit unix
libraries (Proton's layout): the closed-handle case and the wait-all case below
fail, the open-handle case passes. The bug is in the server, not in either
architecture's ntdll.

Cause (server/inproc_sync.c, dlls/ntdll/unix/sync.c)
- A client that waits on an in-process object holds only a duplicate of the
object's ntsync fd, received through the get_inproc_sync_fd request. It holds
no server handle or reference to the object. The client-side cache keeps that
fd open while the wait is in progress (the comment above struct inproc_sync in
dlls/ntdll/unix/sync.c describes the object outliving its handle by way of the
client holding a handle, but nothing holds one).
- So when the process closes its last handle, the server's mutex object is
destroyed right away, and inproc_sync_destroy() takes the in-process sync off
the inproc_mutexes list and closes the server's fd.
- When a thread dies, abandon_inproc_mutexes() sends NTSYNC_IOC_MUTEX_KILL to
every mutex on that list, and only to those. The kernel object is still alive
through the waiter's fd and still records the dead thread as its owner, but no
kill reaches it. The driver abandons a mutex only through that ioctl on that
object, so the mutex stays owned by a thread id that no longer exists, and the
waiter never wakes.

On Windows the kernel keeps the mutants a thread owns on the thread object, so
termination abandons them whatever handles exist: "If a thread terminates
without releasing its ownership of a mutex object, the mutex object is
considered to be abandoned. A waiting thread can acquire ownership of an
abandoned mutex object, but the wait function will return WAIT_ABANDONED to
indicate that the mutex object is abandoned."
(https://learn.microsoft.com/en-us/windows/win32/sync/mutex-objects)

Fix approach that worked (prototype on my machine)
- In inproc_sync_destroy(), when the sync is a mutex and NTSYNC_IOC_MUTEX_READ
reports an owner, keep the fd instead of closing it, on a list of orphaned
mutexes.
- abandon_inproc_mutexes() sends the kill to the orphans as well as to the live
list, then reads each orphan and closes the ones that are no longer owned. The
same pruning runs when an orphan is added, so the list only ever holds mutexes
that were closed while owned and whose owner has neither released them nor died
since.
- Unowned mutexes are closed as before, so the common case is unchanged. With
this, kernel32:sync passes (0 failures in several runs), ntdll:sync is
unchanged, the waiter in the timed sequence returns WAIT_ABANDONED within 0.1
ms of the terminate, and the server's fd count is the same after repeated runs
as before them.
- That alone leaves a gap: a mutex that is unowned when its last handle is
closed is still forgotten, and a wait-all that was pending on it can acquire it
later; if that thread then dies, the mutex is again not abandoned. A probe of
that sequence (two threads in wait-all calls on the mutex and an unset event
each; close the handle; set one event; terminate that thread; set the other
event) fails on stock Wine; on Windows 11 the second thread gets
WAIT_ABANDONED_0 as soon as its event is set (3 of 3 runs).
- Second part, for that gap: the client-side cache entry of an in-process sync
counts its users, so NtClose() (and NtDuplicateObject() with
DUPLICATE_CLOSE_SOURCE) can tell when another thread is still using a mutex. It
then passes an id for the entry with the close_handle request, the server keeps
a reference to the in-process sync under that id (which keeps the mutex on the
list it kills at thread death), and the entry's last user drops it with a new
request; a process's records go with the process. Handles closed with no other
user, the common case, involve no extra work. This is the design the comment
above struct inproc_sync in dlls/ntdll/unix/sync.c describes, done with a
serial number instead of a handle. With both parts the wait-all probe passes on
Wine as on Windows (30 of 30 runs each for 64-bit and 32-bit builds of the
probe; kernel32:sync and ntdll:sync unchanged; the server's fd count is the
same after the runs as before).
- The second part adds a field to two requests and one new request, so it bumps
the server protocol version. In a build with separate 32-bit unix libraries,
the 32-bit unix ntdll has to be rebuilt along with the 64-bit one and the
server, or 32-bit processes cannot connect (the protocol version check in
ntdll). Space Marine 2, for one, starts through a 32-bit launcher stub on every
launch even though the game is 64-bit, so a build that only updated the 64-bit
side would stop the game from starting.

Note on provenance: the investigation was done with the help of an LLM tool,
and the prototype fix and the timed probe were generated with it. Per the Clean
Room Guidelines I am not submitting that code; this report describes the
problem and the approach so that the fix can be written by a Wine developer.
Measurements were taken on my machine (Bazzite 44, kernel 7.2.7 with ntsync,
Ryzen 7 9800X3D) and on a Windows 11 VM on it; the test binaries were built
from Proton Experimental's Wine tree (adea6cb), whose server/inproc_sync.c
matches master apart from Proton's fsync code. The kernel32:sync and ntdll:sync
runs are the 64-bit test binaries; the timed probe was run as both a 64-bit and
a 32-bit program.

Found while running kernel32:sync for bug 60397; unrelated to it otherwise.

-- 
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.