Re: [PATCH v3 0/1] MR499: server: Allow cancelling alerted asyncs.

[Jinoh Kang (＠iamahuman)]

> > I'd propose that the constituent conditions of each flag be refactored into separate state variables first. Paradoxically, it may even help combine two or more binary flags into an enum as the semantics become more simpler.
>
> Probably, but it's not clear to me how exactly to improve them?

I'm yet to come with a concrete proposal, but the basic idea is that they'll be decomposed until some subset of components can be condensed into a phase enum (or the like).

We can already do this with `signaled` and `unknown_status`, for (a not really good) example.
> > * `direct_result`: from Wine codebase, `a flag if we're passing result directly from request instead of APC`.
>
> The idea behind direct_result is that there are two paths for asyncs:
>
> * Requests that can be filled immediately. The idea here is then that we can do this in two server requests: one to send and/or retrieve the data, then the client can set iosb and/or output buffers, and then the second server request is done to trigger completion after the iosb and output buffers have been filled. This is the "direct_result" case. The prototypical example is a named pipe ioctl. The first request varies, but as you've observed, it's basically anything that calls create_request_async(), and then the request handler needs to not pend the async.

Nit: "pend the async" may need some clarification.  There are two possible interpretations to "pend an async":

1. Queueing an async for deferred asynchronous completion.
2. Simply calling `set_async_pending(async)`.

For case 2, `direct_result` can still be true if we terminate the async immediately afterwards.  This is akin to calling `IoMarkIrpPending(Irp);` and immediately following it with a call to `IoCompleteRequest(Irp, IO_NO_INCREMENT);`.

I'll assume that you meant case 1 here.
> * Requests that can't be filled immediately. In this case we use an APC to notify the client when the request is filled. This is the "!direct_result" case. The APC is a callback in a sense, but you can also look at it as two separate requests. As with the direct_result case, we need the server to return the iosb and output buffer (the APC arguments), then the client fills them, then it sends another request to the server (returning from the APC) that lets the server queue completion.
> > * The async was created via create_request_async(), and
>
> The other path is register_async, and I think that should just go away. The idea of register_async is that the client tries to do i/o, gets EWOULDBLOCK, and then sends an async to the server. It was once used for sockets, and now I believe only applies to some "special" unix file types: serial, char device, mailslot, probably also FIFO. Besides being kind of redundant, it has a fundamental problem, as we've discovered with sockets: inauspiciously timed I/O on a "ready" file can jump ahead of queued I/O. I think in general we want to replace it with something that looks like the socket path.

I think that depends on whether the file object serializes I/O requests.  If the file object can complete I/O requests in a nondeterministic order, then I don't think the approach merits extra round-tripping overhead.
> For the time being, though, you can think of register_async as being a "normal" request async that has initial_status pre-initialized to STATUS_PENDING.
> > * The async handle has never been waited on (wait consumes the flag), and
>
> Well, a *successful* wait consumes the flag; the difference is probably a bit pedantic, but if you're trying to fully understand asyncs, probably does still matter :-)

Right, that bit it is also (inadvertently) inconsistent with my prior remark that explicitly mentions "satisfied" wait.  A notable case where waiting for an async handle can be interrupted is when alertable I/O is specified.

Side note: what if `server_read_file` is blocked in an alertable wait for an async handle with `async->unknown_status == 1`, and the blocked thread is sent a user APC?  I suppose the thread should not be alerted *until after* the driver returns from the dispatcher.
> That said, this logic seems questionable. It looks kind of like it exists to avoid calling async_set_result() twice, but that shouldn't be possible anyway. It should only be possible to reach async_satisfied() once. I guess a sufficiently misbehaving client could clone the async handle and then wait on it twice, though... and probably async_set_result() doesn't protect against being called multiple times...

Yeah, `async_set_result` may need its own flag; perhaps `fully_completed` from your suggestion, `settled`, `finalized`, `completion_sent`, or `completion_notified`.
> > * Any of:
> >   * async_handoff() has not been called, or
>
> I'm not sure this is a meaningful state. Until we get to async_handoff() the async isn't *really* initialized. Honestly, it wouldn't be the craziest thing in the world to defer async creation until that point, possibly by building up some sort of parameters structure that we pass to read/write/ioctl fd callbacks instead of a whole async object. But I don't know, that may be more trouble than its worth.

`queue_async(q, async);` may precede `async_handoff(async, ...);`.  Since `queue_async` needs the async to be materialized, we would need to reorder these two calls.
> I suppose we could introduce some sort of bool helper instead of having a direct_result field. It seems relatively clear to me, or at least, not worse than its constitutent logic, but if someone less used to the code has a suggestion, that does deserve wait. It's worth pointing out though that given that part of the condition for whether something is a direct result is "was it ever unknown_status", we'd need another bool flag anyway. I guess then the logic would be as simple as "!was_unknown_status && initial_status != STATUS_PENDING".

Rather, I want `direct_result` (or a similar flag) to mirror the client state: to be precise, whether the client side has went through the synchronous IOSB filling code section at the time of last state synchronization.  Ideally, it should be strictly independent of the server or I/O operation state, so that the async code can be decomposed into an I/O state handling component and a client notification handling component.
> "signaled" is a kind of similar case, where you could in theory break it down into a collection of simpler bits, but not all of those simpler bits currently exist. Fundamentally, "signaled" is "blocking ? fully_completed : !unknown_status".

Er, to be exact, it's `blocking ? fully_completed : (!unknown_status && initial_status_sent_to_client)`, where `initial_status_sent_to_client = irp_completion_deferred || fully_completed`.
> > * `async_wake_obj(async)` has been called when `async->unknown_status = 0`, or
>
> async_wake_obj() is horrid, because it's basically a leaky abstraction. There should be a better way to organize this, but I'm not really sure how. It kind of looks like we should wake up the async in async_set_initial_status(), but the problem is that ntoskrnl can either return the initial status of an IRP by itself but leave the IRP pending, or it can return the initial *and* final status of a complete IRP, and for the sake of efficiency I don't think we want to wake up the async in async_set_initial_status() in the latter case. That would basically result in a SIGUSR1 when we could just return the APC by interrupting the wait. It would *work* but it would be slower.

It does not work, since the IOSB has not been filled by `APC_ASYNC_IO` from `set_irp_result` at the time of wake-up.

What I suppose is that `async_wake_obj` should just be renamed to `async_commit_pending_status` or something similar.
> > * `async_handoff(async, ...)` has been called with `get_error() != STATUS_ALERTED && (async->pending || !NT_ERROR(get_error()) && (get_error() != STATUS_PENDING || async->iosb->status != STATUS_PENDING)`.
>
> What makes this ugly, I think, is that async_handoff() just has a really complex convention, that overloads STATUS_PENDING in multiple ways. That is really what ought to be changed; we should just be more explicit about what we want in the request handler. dd58bf9ce23 was a change in this direction, and I think we can do more.

We can add `STATUS_ALERTED` to the list. I don't think `STATUS_ALERTED` has an intrinsic meaning in the Windows I/O subsystem anyway.
> > * (Regardless of `async->blocking`) `async_set_result(async, status, ...)` has been called with `async->terminated && (!async->alerted || status != STATUS_PENDING)`.
>
> Which is to say: the client finished processing an APC and didn't restart the async.

An interesting bit in the condition above is that `STATUS_PENDING` is interpreted *differently* depending on the value of `async->alerted`.  I think it's a little inconsistent; we should just have a "restart" boolean flag and fail entirely if `restart && !async->alerted`.
> > * If `!async->blocking`, it tries to signal as soon as the initial status is known, *unless* the wait handle is being closed, in which case it doesn't bother to touch the flag at all.
>
> Yeah... that's maybe not great for clarity, but on the other hand, it's also true that "signaled" won't ever be read in that case...
> > Independent of above, I'm thinking about introducing a new `completed` flag and replacing `terminated` with `completed || alerted`.
>
> That sounds like probably a good idea, yeah.

-- 
https://gitlab.winehq.org/wine/wine/-/merge_requests/499#note_31772