On Wed Oct 19 12:08:10 2022 +0000, Giovanni Mascellani wrote:
What I would try to do (and maybe one day I'll have time to actually try), instead of sidestepping pthreads calling cleanup handlers, would be to arrange things so that pthreads can actually call cleanup handlers and they do they right thing. I still have to think this more, but I guess it should be attainable if we arrange the syscall stack so that each userspace entry/exit pair looks like a usual frame (maybe the task of cleaning it up could be delegated to a cleanup handler as well, if it is beneficial). I think your proposal of avoid using the initial thread and only run Windows code in threads created by us would make everything easier. For one thing all threads would have the same stack layout (i.e., they would have a stack allocated and decided by us even from the point of view of pthreads; now, instead, for the main thread we are pivoting to a different stack without telling pthreads, which is why we need `exit_frame` I think).
We could maybe arrange the syscall unwind information so that `pthread_exit` would skip the PE frames, but `pthread_exit` on normal thread exit is only one side of the problem.
The other side is when exception happen in unix code, in which case we jump back to user code while leaving pthread cleanup handlers in place. If that happened, and even if everything else runs fine, calling pthread_exit on thread exit becomes dangerous.
To address that scenario we would have to partially unwind the stacks when returning to user mode, and call any pushed handler. This isn't possible with the C cleanup handlers ABI, which are only registered to some internal pthread structures. It would be possible if the code was in C++, where the handlers are registered as unwinding personality functions.