Hi Mike, all,
I have thought some more.
Having understood the problem with the possibly duplicate process fds, it is clear that in order to treat multiple simultaneous requests correctly, there must be some server-side support on a per-IO-object basis.
With the current implementation as a guideline, I see only one possibility to do this:
The server maintains two semaphores for async reading and writing for each I/O object (file, socket, comm port...) that has the OVERLAPPED flag set. The semaphores have a max count of 1 for FIFOS and some larger number for random-access files (this will be set by object-type-dependent functions).
The async_private struct has a Semaphore field. When the client schedules an async request, it gets the suitable semaphore handler (read/write) of the I/O object from the server and stores it in the async_private struct. The async_handler function does not start actual I/O before acquiring the semaphore, and releases it when finished. The handler is in IO_PENDING state until it gets the semaphore and remains in that state after obtining access until it releases the semaphore.
This provides a rather elegant framework for limiting the number of concurrent readers and writers. I am not sure, though, if the async_handler functions are allowed to wait for semaphores ??? Perhaps we'd need to use Unix semaphores ?
Any other approach I've thought of so far seems to imply a lot of client-server interaction and mangling in the NtCurrentTeb()->pending_list. Please notify me if you have better ideas.
The other possible solution would be to go back to the server-centric implementation of 8 months ago, or something similar. Most likely the NtCureentTeb()->pending_list would have to be dropped in that case, and server-side communications used instead.
However, I assume you have had good reasons to abaondon the old server-centric implementation, so I'm reluctant to go that way.
There is another concern, though. You wrote:
When the thread waits upon any server object, all (unix) fds with pending overlapped operations are polled on. This is a minor problem in the design; if the thread never waits upon an object the overlapped can never complete, however the thread must wait upon the overlapped's hEvent to check whether it has completed...
AFAICS, there will not even be an I/O request unless the client waits on something (at least in ReadFileEx()). I wonder if this can cause the advantages of async IO to be lost if a a long time elapses before the client waits on anything. Definitely, if the first wait the client issues is on the async request itself, the handler will have done nothing in the meantime and there is no real asynchronicity any more. Meanwhile all arriving data must be buffered by the kernel. This is probably no problem on a serial port, but what if you're dealing with a fully loaded ethernet connection or a local socket with a lot of traffic?
Anyway, That's probably a minor issue.
Please tell me if you think the semaphore idea is feasible.
Martin
-
Martin Wilck