http://bugs.winehq.org/show_bug.cgi?id=60192 Bug ID: 60192 Summary: ntsync: lost wakeup causes permanent deadlock on infinite-timeout WAIT_ANY Product: Wine Version: 11.0 Hardware: x86-64 OS: Linux Status: UNCONFIRMED Severity: normal Priority: P2 Component: ntdll Assignee: wine-bugs@list.winehq.org Reporter: strikeralt40@gmail.com Target Milestone: --- Distribution: --- Summary Under heavy multi-threaded load, ntsync intermittently and permanently loses a wakeup for an infinite-timeout wait (NTSYNC_IOC_WAIT_ANY with timeout = U64_MAX). The waiter never wakes even though the event it waits on is set tens of thousands of times afterward. The task remains blocked in schedule_hrtimeout_range_clock() forever, deadlocking the entire application. Reproduced with Grand Theft Auto V (job-system synchronization) on kernel 7.1.8-1-cachyos. The same workload runs indefinitely with esync/fsync, and the same game runs indefinitely on ntsync once it gets past the freeze window, so the failure is a rare, intermittent lost-wakeup race, not a guaranteed bug. Environment - Kernel: 7.1.8-1-cachyos (CONFIG_NTSYNC=m, ntsync loaded, /dev/ntsync rw-rw-rw) - ntsync source: drivers/misc/ntsync.c, identical logic in v7.1 and current master (the only master difference is timens_ktime_to_host, which does not touch the sleep path) - Wine: cachyos-11.0 (Wine 11.0), 64-bit, NTSYNC_IOC_WAIT_ANY - Application: Grand Theft Auto V v1.69, job-system spawns dozens of worker threads - strace 7.0 attached to all GTA5.exe threads during a live freeze Reproduction Run GTA V (or any app that heavily uses ntsync events with infinite waits across many threads) under Wine with ntsync. The job system freezes at a random time — minutes to ~2 hours in. Symptoms at freeze: - Main thread spins forever in GTA5.exe+0x165F5B5 (job-sync spin loop) - Job counter frozen at 30 (0x1e) - Workers keep calling NTSYNC_IOC_EVENT_SET on the event the main thread waits on, but the main thread never wakes. The freeze is permanent (observed 30+ minutes). esync/fsync do not reproduce it. Trace evidence (215 MB strace capture, all GTA5.exe threads) Key ioctl counts during the freeze: ioctl count NTSYNC_IOC_EVENT_SET total 506,041 NTSYNC_IOC_EVENT_SET on fd 733 243,225 NTSYNC_IOC_WAIT_ANY issued 841,093 NTSYNC_IOC_WAIT_ANY still unfinished at capture 622,071 threads stuck in WAIT_ANY at capture end 694 The main thread's ioctl(10, NTSYNC_IOC_WAIT_ANY, ...) was issued once and never resumed (<unfinished ...> persists to end of capture), while fd 733 (a ntsync event) was set 243,225 times by worker threads. The signal was delivered to the kernel's wake path, but the waiter's wake_up_process() either raced with its schedule() or was absorbed, and the thread slept forever. Root-cause analysis ntsync_schedule() (drivers/misc/ntsync.c, both v7.1 and master): static int ntsync_schedule(const struct ntsync_q *q, const struct ntsync_wait_args *args) { ktime_t timeout = ns_to_ktime(args->timeout); clockid_t clock = CLOCK_MONOTONIC; ktime_t *timeout_ptr; int ret = 0; timeout_ptr = (args->timeout == U64_MAX ? NULL : &timeout); do { if (signal_pending(current)) { ret = -ERESTARTSYS; break; } set_current_state(TASK_INTERRUPTIBLE); if (atomic_read(&q->signaled) != -1) { ret = 0; break; } ret = schedule_hrtimeout_range_clock(timeout_ptr, 0, HRTIMER_MODE_ABS, clock); } while (ret < 0); __set_current_state(TASK_RUNNING); return ret; } The wake path (try_wake_any_event): if (atomic_try_cmpxchg(&q->signaled, &signaled, entry->index)) { if (!event->u.event.manual) event->u.event.signaled = false; wake_up_process(q->task); } For an infinite wait, timeout_ptr == NULL, so schedule_hrtimeout_range_clock(NULL, ...) is reached. In kernel/time/sleep_timeout.c: if (!expires) { schedule(); return -EINTR; } For the NULL (infinite) case, schedule_hrtimeout_range_clock() performs a plain, unconditional schedule() — it does not re-check the condition after waking, and more importantly it does not participate in the wakeup protocol the same way the non-NULL path does (hrtimer_sleeper_start_expires() + schedule() + hrtimer_cancel()). The racy window: 1. Waiter runs set_current_state(TASK_INTERRUPTIBLE), reads q->signaled == -1 (not yet signaled), is about to call schedule_hrtimeout_range_clock(NULL). 2. Waker wins atomic_try_cmpxchg(&q->signaled, -1, index), then calls wake_up_process(q->task). 3. The waiter is still on the runqueue at this point (it has not yet entered schedule()), so try_to_wake_up() takes the on_rq && ttwu_runnable() fast path in kernel/sched/core.c — it "wakes" the task without actually placing it on the runqueue, because it is already there. 4. The waiter then calls schedule(); with prev->state == TASK_INTERRUPTIBLE, __schedule() dequeues it and it sleeps. 5. q->signaled is now non--1, so every subsequent NTSYNC_IOC_EVENT_SET fails the atomic_try_cmpxchg(&q->signaled, -1, ...) and never calls wake_up_process() again. The wake is permanently lost. This exactly matches the trace: 243,225 EVENT_SET calls on the event, none of which can re-wake the thread, because the cmpxchg only succeeds once and the thread never consumes the signal. esync/fsync are immune because futex FUTEX_WAIT/FUTEX_WAKE and eventfd wakeup have stricter semantics than wake_up_process() on a task that has set_current_state(TASK_INTERRUPTIBLE) but has not yet called schedule(). Suggested fix directions 1. In ntsync_schedule(), for the infinite (timeout_ptr == NULL) case, use a loop that re-checks atomic_read(&q->signaled) after every schedule()/wakeup instead of relying on schedule_hrtimeout_range_clock() returning; and/or use wait_event_interruptible()-style semantics so that a condition satisfied before schedule() is observed after it. e.g. for timeout_ptr == NULL, replace with: while (!signal_pending(current) && atomic_read(&q->signaled) == -1) { set_current_state(TASK_INTERRUPTIBLE); if (atomic_read(&q->signaled) != -1) { ret = 0; break; } schedule(); } (which mirrors what wait_event_interruptible() does internally) so that a wakeup that "lands" before the task actually sleeps is not lost. 2. Alternatively, in the wake path, ensure that once q->signaled has been set, a subsequent wake_up_process() is issued even if the cmpxchg fails — i.e., make the wake resilient to the race by not gating the wake strictly on the cmpxchg winning. 3. Consider whether schedule_hrtimeout_range_clock(NULL) should be avoided in favor of an explicit bounded timeout converted from U64_MAX semantics, so the timer-based path (which handles spurious-wakeup correctly) is always used. Files inspected - drivers/misc/ntsync.c (v7.1.8 and master — identical sleep logic) - kernel/time/sleep_timeout.c (schedule_hrtimeout_range_clock, NULL/infinite path) - kernel/sched/core.c (try_to_wake_up, ttwu_runnable, __schedule) - strace capture /tmp/opencode/ntsync_trace.txt (215 MB, all GTA5.exe threads) Impact Any Windows app using ntsync events/mutexes with infinite waits under heavy multithreading can freeze permanently. This is not GTA-specific; GTA V is simply the test case that exercises the race often enough to observe it. The bug is present in current mainline ntsync (verified against master as of the report date). -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.