While working on https://gitlab.winehq.org/wine/wine/-/merge_requests/8572 I discovered that Windows, unlike our implementation, prioritizes user APC presence over satisfied wait conditions. Trying to fix that uncovered a couple of places covered by the tests which depend on the present WaitFor behaviour, one of those is the alertable synchronous IO.
While looking into this part turned out alertable synchronous IO has more issues unrelated to this user APC precedence:
1. Currently wait_async() performs alertable wait for FILE_SYNCHRONOUS_IO_ALERT, and if user APC is queued that exits IO function at once. While not yet completed async is left behind and will write IOSB to then-random memory location once it is finally finished. Also, we return STATUS_USER_APC from NtWaitFor... performed in wait_async(), Windows instead returns STATUS_CANCELLED (across all the Windows versions), indicating that async IO was canceled first. That works the same way with NtCancelSynchronousIoFile. That is unrelated to fixing NtWaitFor..
2. With NtWaitFor... preferring user APC result, alertable wait in wait_async() gets interrupted with user IO completion APC queued by us (if the APC is passed to IO function) while completing APC, so wait_async() ends with STATUS_USER_APC even if there was no side user APC queued.
3. On Windows IO completion user APC (queued by system on async completion) with alertable blocking IO file is executed on IO function exit, as also shown in my added tests. That is not happening now, because wait resolves first by signaled async and there is nothing to process the queued APC.
4. Up to date Windows 11 also sets IOSB on canceled sync read (both alertable and not). That's not only with the read interrupted by APC but also for NtCancelSynchronousIoFile. This looks like new behaviour, none of Testbot machines fill IOSB in my added tests. But my local Win11 24H2 does. That behaviour was on Wine was prevented by the check in async_terminate() introduced by commit 5af74129bd5d5f5c37ba25393da74c077fcb4676. I checked kernel32/tests/volume.c on my Win11 24H2 and confirmed that the tests still succeed on it. SO it doesn't look like the behaviour changed for all the error return cases, but it is now the case for blocking canceled asyncs at least. This part is concerned in the last patch "server: Set IOSB for cancelled asyncs in async_terminate()."
The general idea behind fixing the most part of it (pp1-3, while p. 4 is mostly independent) is to get rid of alertable wait in wait_async(). It looks straightforward to me that we'd better manage special async object signal state properly, so it is only signaled once APC is concluded one or another way. While if that is alertable IO and user APC is present we should signal the async object; while wait_async() on the client can check for user APCs on exit, on the way fixing p. 3.