Re: [PATCH v2 0/1] MR1065: ntdll: Add CFI unwind info to __wine_syscall_dispatcher (x86_64)

[Florian Kübler (＠florian-kuebler)]

> BTW, if the issue is only that the CFA is supposed to stay constant within a CFI procedure, would it work if we were splitting the dispatcher with some jumps / call / fake CFI procedures, at the points where we need to change its register?

My point is, that the issue is actually not about CFA not being constant, but that CFA has a semantic that many unwinding tools/libraries rely on, i.e. that CFA is suppose to be the value of the stack pointer at the caller, right before the call. In particular, CFA is suppose to be a value that would be a valid stack pointer.

In your change, however, CFA gets later set to be a pointer into syscall_frame.
> With GDB and with the two other changes I mentioned, it fully works yes.

Ah, that is good to know, thanks. Apparently gdb's unwinder works slightly different, than the used we looked at. Also, I am surprised it actually can unwind Windows PE/Coff-based frames.
> I'm a bit surprised that you don't need the flags push/pop change, as I believe it corrupts the return address on the stack, but maybe the unwinder has other ways to get it back.

Interesting. I am wondering why the `pushfl` instruction corrupts the return address. If this would be the case, wouldn't that not lead to actual runtime issues when the dispatcher returns?
Maybe the reason why we don't see an issue with that, is that we are actually setting the CFI rule for the return address based on `syscall_frame` right away?
> Yes, that's what I've been told (maybe even *you* did).

No, that wasnt me :smile:, it's actually my first PR. Also note, that I am not a wine expert, so please forgive me, if I miss something obvious.
> Now the other part where unwinding doesn't work well is with the other side of syscalls, with KiUserCallbackDispatcher callbacks.

Unfortunately, I am not aware of the `KiUserCallbackDispatcher`. So I can't tell if there is more work to be done/how this could be fixed.

We basically looked at reported unwinding errors in Orbit, and discovered the missing unwind info (CFI) being the root cause of most of them. With this, and some adjustments to our unwinder to cope with certain corner-cases of PE/Coff files, we ended up with less than 1% of unwinding errors.

Actually, another issue that kept us quite busy is, that the `kernel_stack` is not allocated on the "normal"/"user" stack. That apparently used to be the case before Wine 7. This way, Linux performance collector tools, such as perf_event_open, fail to collect the "user-space" part of the stack. For debugging this is fine, as the program execution is halted and we can just read the data from memory, but for Profiling, we needed to massively work around that.
Does someone know about the reason for the two separated stacks? Would it be maybe possible, to actually allocate that "kernel_stack" at the normal stack?