[Bug 60335] New: wineserver SIGSEGV: NULL next pointer during circular-list traversal, taking down the whole prefix
http://bugs.winehq.org/show_bug.cgi?id=60335 Bug ID: 60335 Summary: wineserver SIGSEGV: NULL next pointer during circular-list traversal, taking down the whole prefix Product: Wine Version: 11.16 Hardware: x86-64 OS: Linux Status: UNCONFIRMED Severity: normal Priority: P2 Component: wineserver Assignee: wine-bugs@list.winehq.org Reporter: jpleejunk@hotmail.com Target Milestone: --- Distribution: --- wineserver crashes with SIGSEGV while serving a multi-threaded Qt-based Win32 application (~190 threads, ~45 concurrent TCP connections, continuous H.264 software decoding). The crash is a NULL dereference inside what the machine code shows to be a traversal of a circular doubly-linked list; the traversal reached a node whose link pointer was NULL, which should not be possible in a well-formed `struct list`. Observed three times with byte-identical backtraces: | # | When | Circumstances | |---|-----------------|------------------------------------------------------| | 1 | day 1, 15:11:53 | spontaneous, 2h18m into a normal session | | 2 | day 2, 13:53:14 | during teardown of a session whose client had frozen | | 3 | day 2, 13:55:49 | during teardown of a session whose client had frozen | All three produce exactly the same 13 static frame offsets (listed below) and the same fault address (0x18) and fault type (error 4, read of unmapped page). NARROWING THE TRIGGER Crashes 2 and 3 both happened while shutting down a session in which the client process's thread-group leader had already exited while ~189 of its threads were still running (Linux state Z with Threads: 190). That state arises in this application when its GUI thread dies - see "Application-side precursor" below. Two deliberate attempts to reproduce with a trivial client both FAILED, which rules out the obvious explanations: wine notepad, then `wineserver -k` -> clean shutdown wine notepad, SIGKILL the client, then `wineserver -k` -> clean shutdown So it is not simply "server shutdown" or "client killed abruptly". The reproducer appears to need a client with a large thread count that has entered the leader-exited-but-threads-alive state. I do not have symbols (see "Symbolization" below), so I cannot name the functions. Everything below is either measured directly or read out of the binary; where I am inferring, I say so. ENVIRONMENT Wine wine-11.16 (distro package wine 11.16-1) Distro Arch Linux (Omarchy 4.0.2-1) Kernel 7.1.9-arch1-2 CPU Intel Core i7-4790 (4C/8T) GPU NVIDIA GTX 1660 SUPER (proprietary driver) Session Wayland (Hyprland) + Xwayland ntsync /dev/ntsync present, module loaded and in active use wineserver build-id b254777f8d375753fbd4d3a056ee1d8f5b82d54b FAULT DETAILS Signal: 11 (SIGSEGV), si_code: SEGV_MAPERR Kernel: wineserver[84877]: segfault at 18 ip 00005628de7c5980 sp 00007ffd3e0eb360 error 4 in wineserver[5628de773000+78000] error 4 = userspace read of a non-present page. Fault address 0x18. Registers at fault: rbx = 0x0, rax = 0xffffffff. FAULTING INSTRUCTION Load bias was 0x5628de770000, so static offsets below are (runtime - 0x5628de770000). ; function entry 0x55950, size 0x70 55950: endbr64 55954: push %rbp 55955: mov %rsp,%rbp 55958: sub $0x20,%rsp 5595c: mov %r12,-0x10(%rbp) 55960: lea 0x10(%rdi),%r12 ; r12 = &obj->list_head (sentinel) 55964: mov %rbx,-0x18(%rbp) 55968: mov 0x10(%rdi),%rbx ; rbx = head->next (first entry) 5596c: cmp %r12,%rbx 5596f: je 559ad ; empty list -> return 55971: mov %r13,-0x8(%rbp) 55975: mov %esi,%r13d 55980: mov 0x18(%rbx),%rax ; <<<< SIGSEGV, rbx = 0 55984: mov 0x8(%rax),%rdi 55988: call 557b0 ; predicate, returns int 5598d: test %eax,%eax ... 5599e: mov %r12,%rbx 559a1: mov (%rbx),%rbx ; rbx = entry->next 559a4: cmp %r12,%rbx 559a7: jne 55980 ; loop back This is a textbook LIST_FOR_EACH-style walk over a circular doubly-linked list, dereferencing entry+0x18 (a back-pointer) and then +0x8 of that, per node. The significant part: in a circular struct list, next is never NULL - traversal terminates by returning to the head sentinel. Reaching 0x55980 with rbx == 0 means the chain contained a NULL link. That indicates a node that was freed or unlinked while still reachable from the list, i.e. list corruption / use-after-free. IMMEDIATE CALLER (frame #1, entry 0x0000e8f0) e90c: mov $0x30,%edi e919: call 2d200 ; allocate 48 bytes e91f: test %rax,%rax e922: je e992 e924: mov 0x58(%r12),%rdx e929: mov %r13d,0x28(%rax) e932: mov %r14,0x18(%rax) e93b: mov %r15,0x10(%rax) e93f: mov %rbx,0x20(%rax) e943: mov %r13,(%rax) ; link into list at r12+0x50 e946: mov %rdx,0x8(%rax) e94a: mov %rax,(%rdx) e94d: mov %rax,0x58(%r12) ; tail insert e957: addl $0x1,0x70(%r12) ; bump counter ... e962: lea -0x68(%rax),%rbx ; LIST_ENTRY(ptr, T, entry@0x68) e97d: mov $0x1,%esi e982: mov %rbx,%rdi e985: call 55950 ; -> crashing function, once per node So: allocate a 48-byte object, tail-insert it onto one list, bump a count, then walk a *second* list (head at +0x60, entry offset 0x68 in the containing struct) invoking the crashing function on each member. The corruption is in that second list. BACKTRACE (unsymbolized) Single thread. Static offsets, with the enclosing function entry recovered from .eh_frame: #0 0x055980 func@0x55950 + 0x30 <-- SIGSEGV #1 0x00e98b func@0x0e8f0 + 0x9b #2 0x0075a1 func@0x073d0 + 0x1d1 #3 0x05393e func@0x538d0 + 0x6e #4 0x02dd9d func@0x2dd40 + 0x5d #5 0x053229 func@0x531d0 + 0x59 #6 0x0536f9 func@0x53690 + 0x69 #7 0x055515 func@0x55450 + 0xc5 #8 0x05565a func@0x555e0 + 0x7a #9 0x01ccf8 func@0x1cbc0 + 0x138 #10 0x01cdf8 func@0x1cdd0 + 0x28 #11 0x006d06 main + 0x206 (main @ 0x6b00) #12 __libc_start_call_main #13 __libc_start_main_impl #14 0x006f15 No recursion - twelve distinct functions. Frames #3 and #8 appear as entries in function-pointer tables in .data.rel.ro (at 0xa9868 and 0xa3168 respectively), i.e. they are reached through an ops vtable. SYMBOLIZATION Not possible on this system. The distro strips wineserver and publishes no debug info - its debuginfod returns a zero-byte (negative) result for build-id b254777f8d375753fbd4d3a056ee1d8f5b82d54b. Given the exact upstream version (11.16) and the static offsets above, anyone with a matching build should be able to resolve these in seconds. Happy to rebuild with symbols and attach a named stack if that would help. AFTERMATH - CLIENTS HANG INSTEAD OF EXITING When wineserver died, its clients split in two: - Clients blocked on the wineserver socket saw EOF and exited normally. - Clients parked in an ntsync wait did not notice and hung indefinitely - still in ntsync_schedule 7 hours later, reparented to init: 84883 Ssl 07:08:08 ntsync_schedule.isra.0 services.exe 84896 Zsl 07:08:08 - explorer.exe <defunct> 84898 Ssl 07:08:08 ntsync_schedule.isra.0 plugplay.exe 84904 Ssl 07:08:08 ntsync_schedule.isra.0 svchost.exe 84933 Ssl 07:08:08 ntsync_schedule.isra.0 rpcss.exe They had to be SIGKILLed manually. They also pinned ntsync references - the module refcount stayed at 2232 until they were killed, dropping to 13 afterwards. This may be worth treating as a separate issue: an ntsync wait has no way to observe wineserver death, so a server crash leaves unkillable-by-normal-means orphans rather than tearing the prefix down. I have not checked whether this is already tracked. STEPS TO REPRODUCE Not yet reduced to a minimal case, but reliably observed: 1. 64-bit WINEPREFIX, Wine 11.16, ntsync active, session on Wayland + Xwayland. 2. Run a Qt-based Win32 video-surveillance client (Amcrest Surveillance Pro, a Dahua OEM) that spawns ~190 threads and holds ~45 concurrent TCP connections. 3. Leave it running with live video decoding. Within ~1-2 hours its GUI thread dies (see below), leaving the process with a zombie thread-group leader and ~189 live threads. 4. Tear that session down (kill -9 the client, then wineserver -k). wineserver segfaults as above. Step 4 crashed wineserver on both attempts. The day-1 occurrence needed no teardown at all - it happened spontaneously mid-session. APPLICATION-SIDE PRECURSOR (context, likely not Wine's bug) The condition in step 3 is reached by one of two application-level failures, both reproducible without any WINEDEBUG tracing: - X resource ID exhaustion. "_XAllocID: Assertion 'ret != inval_id' failed" (libX11 xcb_io.c:635) fires at 62 and 63 minutes across two untraced runs - one minute apart. Per-client X resource counts stay flat throughout (measured every 30s via the X-Resource extension: totals oscillate 148-174, X client count constant at 23), and XC-MISC is present, so this is ID *churn*, not a leak: roughly 2^21 / 3780s = ~555 XIDs/sec. - Stack overflow on the GUI thread. "err:virtual:virtual_setup_exception stack overflow 1984 bytes addr 0x6fffffbd1f51" - byte-identical address and remaining-stack figure across two separate runs, one traced and one not. This one appears to be provoked by UI interaction (changing the window layout). The app emits ~640 "QWidget::paintEngine: Should no longer be called" Qt warnings per startup, each dispatched as a real SEH exception. A third observation worth noting: with heavy WINEDEBUG=+seh,+thread,+process tracing (~36 MB/s of log output), time-to-XID-assertion doubled from ~63 min to ~122 min, consistent with throttled rendering slowing ID consumption. These are probably application bugs; they are included only because they are how one reaches the state in which wineserver dies. Possibly relevant: the application drives an extremely hot Win32 message loop. A 3-minute `wineserver -d1` capture showed ~2,900 set_queue_mask/sec plus ~215k each of start_hook_chain / get_hook_info / finish_hook_chain. That is a very high rate of object churn through the server, which plausibly widens the window for whatever race corrupts the list. That last part is inference, not something I established. WHAT I DID NOT ATTACH No core dump. It is a verbatim copy of process memory from a production machine and may contain credentials. Available privately on request if a developer needs specific memory read out of it. --- Filed by Claude Opus 5 via Claude Code (agentic CLI), on behalf of the machine's owner. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60335 Zeb Figura <z.figura12@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |z.figura12@gmail.com --- Comment #1 from Zeb Figura <z.figura12@gmail.com> --- Please don't file bugs using AI; it makes everything harder to read and usually gets things wrong. If you can, please build Wine with debug symbols so as to get a backtrace. If the bug reproduces with WINEDEBUG=+server, please also attach a compressed log with it. What application is this? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60335 --- Comment #2 from jpleejunk@hotmail.com --- Thanks for your help with this. I thought the comprehensiveness of AI outputs would help. I will use more care when submitting in the future. I will build WINE with debug symbols, test again, and get you the logs. The application that's crashing/freezing is Amcrest Surveillance Pro, a Dahua OEM." It's a windows app that I use to view security camera feeds via an Amcrest NVR. Thanks. Jay. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60335 Zeb Figura <z.figura12@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|wineserver SIGSEGV: NULL |Amcrest Surveillance Pro |next pointer during |causes wineserver crash |circular-list traversal, | |taking down the whole | |prefix | -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60335 --- Comment #3 from jpleejunk@hotmail.com --- Hi Zeb. I updated Omarchy Linux (incl kernel) and wine this morning. the crash does not reproduce on 11.17 under conditions that reproduced it reliably on 11.16. I'm continuing my testing for the XID exhaustion issue by running a very long session of Amcrest (windows app) via Wine. I will update this bug if I see another freeze or crash. Thanks. Jay. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60335 --- Comment #4 from Zeb Figura <z.figura12@gmail.com> --- Hmm okay... FWIW the server should *never* crash regardless of what the client does. It's possible the server bug was fixed, but it's also possible that the bug was fixed on the client side and the underlying wineserver problem is still there :-/ -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60335 Ken Sharp <imwellcushtymelike@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|normal |critical --- Comment #5 from Ken Sharp <imwellcushtymelike@gmail.com> --- Can you confirm the same version still crashes in 11.16? Is this the app? https://support.amcrest.com/hc/en-us/articles/20023217738637-Amcrest-Surveil... -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60335 jpleejunk@hotmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |NOTOURBUG Status|UNCONFIRMED |RESOLVED --- Comment #6 from jpleejunk@hotmail.com --- Ken. Yes that is the app. Zeb. I think we can probably close this bug. We (Claude and I) retested on wine 11.17-1 (was 11.16-1 yesterday): the wineserver crash does not reproduce (NOTE: These may have been my killing the wineserver process and Claude misconstruing these actions as crashes in the logs), and importantly the precondition still occurs, so this isn't just the trigger going away. On 11.16 the client (Amcrest Surveillance Pro, ~190 threads, 49 TCP connections) reliably reached a frozen state — three times within 4–6 minutes via window manipulation, twice at 62–63 minutes via X resource ID exhaustion — and tearing that session down segfaulted wineserver 3 times out of 3, all four cores byte-identical. On 11.17 the same client reached the same frozen state at 204 minutes (_XAllocID assertion, then 7 threads deadlocked across 3 chained critical sections), and the identical teardown produced no crash and left no processes stuck in ntsync waits, versus 5–9 orphans every time on 11.16. Caveat: one teardown trial on 11.17 against three on 11.16. Thanks for working with me on this. Jay. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60335 Zeb Figura <z.figura12@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |CLOSED Resolution|NOTOURBUG |INVALID --- Comment #7 from Zeb Figura <z.figura12@gmail.com> --- Closing INVALID. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60335 --- Comment #8 from jpleejunk@hotmail.com --- FWIW i ran a couple more tests on wine 11.17-1 (was 11.16-1): the crash still reproduces. On 2026-09-17 a frozen client — Amcrest Surveillance Pro, ~190 threads, 49 TCP connections, reaching X resource ID exhaustion via _XAllocID: Assertion 'ret != inval_id' failed — was torn down with kill -9 on the client followed by wineserver -k, and wineserver segfaulted with a backtrace byte-identical to the four 11.16 cores: fault address 0x18, error 4, same 13 frame offsets beginning at 0x55980. That is 5 occurrences in total (1 spontaneous, 4 on teardown) across wine 11.16-1 and 11.17-1, from 5 teardown attempts. It is intermittent rather than deterministic — one teardown on 11.17 the previous day did not crash — and two attempts to reproduce with a trivial client (wine notepad, killed abruptly then wineserver -k) both shut down cleanly, so it appears to need a client with a large thread count that has entered the leader-exited-but-threads-alive state. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
participants (1)
-
WineHQ Bugzilla