https://bugs.winehq.org/show_bug.cgi?id=52492
--- Comment #13 from Rob F rjfeuerbach@gmail.com --- (In reply to Fabian Maurer from comment #12)
Seems like a sensible approach to me. Although I admit I don't understand why you just add up all the differences, why not sqrt to get the distance between the points? And don't you need flatness * 0.5 like below? Don't take that as critique though, I just don't understand what exactly you're doing.
While we're at it, do you plan to send this to wine? IMHO you got a good fix, or a possibly at least a good start. If there's need for improvement, Wine devs will point it out, and they know a lot more than me.
I chose the taxi-cab distance to (needlessly) avoid more expensive calls. The original test functioned as a stopping condition for the recursive calls, and I was just trying to make the stopping condition less strict, since the original test was failing to stop. The flatness * 0.5 could be added.
A patch with this fix and the test-modification was submitted in Feb 2022, but was not accepted as-it-was. (Adding a test that causes a stack-overflow did not work well.) I'd be happy if someone fixed the issue, with my solution or another one.
In Feb 2022, the problem was only evident on the 64-bit debian test platform, and of course, the 64-bit MacOS I was using.