http://bugs.winehq.org/show_bug.cgi?id=60115 Bug ID: 60115 Summary: d2d_cdt_insert_segment() recurses without a global termination guard, stack overflow on degenerate geometry Product: Wine Version: 11.13 Hardware: x86-64 OS: Linux Status: UNCONFIRMED Severity: normal Priority: P2 Component: d2d Assignee: wine-bugs@list.winehq.org Reporter: Seth2313@gmail.com Target Milestone: --- Distribution: --- IN PLAIN WORDS An application that draws complex vector graphics through Direct2D can kill Wine's main thread. A helper function in Wine's Direct2D triangulation code calls itself over and over without ever stopping when the input geometry is degenerate, until the 1 MB thread stack is used up and the process dies with a stack overflow. There is a safety check in the function, but it only guards a single call level, so it never fires. SUMMARY d2d_cdt_insert_segment() in dlls/d2d1/geometry.c:2287 recurses without a global termination guard. For degenerate geometry (collinear or duplicate vertices) the recursion oscillates between two origins and never terminates, exhausting the thread stack: EXCEPTION_STACK_OVERFLOW (0xC00000FD). The application's main thread dies while its audio threads keep running, so the process turns into a half-alive zombie rather than exiting. ENVIRONMENT Wine: wine-staging 11.9 when analysed; the affected code is byte-identical in 11.13 Application: FL Studio (64-bit) with two or more Serum 2 instances (JUCE/IPlug GUI drawing via Direct2D) d2d1: Wine builtin THE DEFECT static BOOL d2d_cdt_insert_segment(struct d2d_cdt *cdt, struct d2d_geometry *geometry, const struct d2d_cdt_edge_ref *origin, struct d2d_cdt_edge_ref *edge, size_t end_vertex) { for (current = *origin;; current = next) { ... if (<collinear case>) { d2d_cdt_edge_sym(&new_origin, ¤t); return d2d_cdt_insert_segment(cdt, geometry, &new_origin, edge, end_vertex); /* :2312 */ } if (d2d_cdt_rightof(...) && d2d_cdt_leftof(...)) { ... new_origin = *edge; return d2d_cdt_insert_segment(cdt, geometry, &new_origin, edge, end_vertex); /* :2335 */ } if (next.idx == origin->idx) { ERR("Triangle not found.\n"); return FALSE; } } } The only termination guard is "next.idx == origin->idx", which detects a full revolution around THE ORIGIN OF THE CURRENT INVOCATION. Both recursive calls at :2312 and :2335 enter with a NEW origin, so each level starts its guard from scratch. When the traversal cycles between two origins (A -> B -> A -> ...), no level ever completes a revolution, the guard never triggers, and the recursion is unbounded. EVIDENCE 1. Stack overflow (original code). With WINEDEBUG=+seh: code=c00000fd ... stack 0x108c0 (0x10000-0x11000-0x110000) i.e. the 1 MB thread stack is fully consumed. The main thread dies; the process stays alive because its audio threads are unaffected, which presents to the user as a total UI freeze. 2. The non-termination is real, not merely deep recursion. Both recursive calls are tail calls, so they can be rewritten as a loop ("origin = &new_origin; goto restart;") without any change in semantics. With that rewrite the stack overflow disappears and is replaced by an INFINITE LOOP: the thread spins at 100 % CPU with the instruction pointer stable inside d2d_cdt_insert_segment (RVA 0x2e9a3), confirmed with gdb across many samples. That rules out "recursion is merely too deep" and establishes non-termination. WORKAROUND IN USE HERE Tail-call elimination plus a generous iteration budget, bailing out the same way the existing "Triangle not found" path does: --- a/dlls/d2d1/geometry.c +++ b/dlls/d2d1/geometry.c @@ struct d2d_cdt_edge_ref base_edge, current, new_origin, next, target; size_t current_destination, current_origin; + unsigned int iterations = 0; +restart: for (current = *origin;; current = next) { + /* Guard against non-terminating cycles caused by degenerate geometry + * (e.g. collinear/duplicate vertices). The original code recursed here + * and died with a stack overflow; bail out and skip the segment instead. + * A correct segment insertion is O(edge_count), so a generous multiple + * of it is a safe upper bound that legitimate geometry never reaches. */ + if (++iterations > 16 * cdt->edge_count + 0x10000) + { + WARN("Iteration limit exceeded, skipping segment.\n"); + return TRUE; + } with both "return d2d_cdt_insert_segment(...)" calls replaced by "origin = &new_origin; goto restart;". This has been in daily production use since 2026-05 with no visible rendering artefacts. Note this returns TRUE (skip the segment and carry on) rather than FALSE. FALSE propagates up through d2d_cdt_insert_segments() and fails the whole tessellation, which in practice loses far more than the one bad segment. Which of the two is the right upstream behaviour is a call for the d2d1 maintainers - the important part is that the function must terminate. NO MINIMAL REPRODUCER The degenerate geometry is produced at runtime by a closed-source plugin GUI, so I cannot supply a small test case. A targeted test would presumably construct a path geometry containing duplicate and collinear vertices and tessellate it. I am happy to test any candidate patch against the real workload that triggers this reliably. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.