struct async_private needs generalization for Winsock
Hi Mike, all, as I pointed out before, we must change the async_private type such that it accomodates the different needs of normal (e.g. ReadFileEx()) and Winsock (e.g. WSARecv()) async requests. The "generic" functions in scheduler/synchro.c must be able to handle these structures without knowing what type of async_private structure they have been passed. I try to summarize the differences between the two request types here: 1. Winsock async requests have a WSAOVERLAPPED struct rather than OVERLAPPED. Unlike File I/O, the routines complete if any positive number of bytes is read (there's no such thing as BytesToRead). 2. Unlike File I/O, The WSAOVERLAPPED struct is *not* accessed in any way, except for the hEvent field. The other fields do not carry useful information and are, according to the Winsock2 specs, reserved to service providers, i.e. we are not allowed to write to them. 3. Winsock functions take a scatter/gather array of type WSABUF* instead of a single buffer, and a number of WSABUFs instead of the number of bytes to read. 4. The completion routine takes an additional DWORD argument. 5. In the case of WSARecvFrom()/WSASendTo(), the socket address and length parameters passed by the user must be stored and filled in at completion. [These are the differences I've come across so far, there may be more.] I'd therefore recommend the following, similar to the treatment of "struct object" and derived types in the server code: struct async_private; typedef void (*async_handler)(struct async_private *ovp); struct async_private { unsigned int type; /* file/socket, read/write/... */ HANDLE handle; int fd; struct async_private *prev; struct async_private *next; async_handler func; } struct async_fileio { struct async_private async; char *buffer; int count; LPOVERLAPPED lpOverlapped; LPOVERLAPPED_COMPLETION_ROUTINE completion_func; } struct async_winsock { struct async_private async; LPWSABUF wsabuf; int n_wsabufs; LPWSAOVERLAPPED overlapped; LPWSAOVERLAPPED_COMPLETION_ROUTINE completion_func; struct WS_sockaddr* addr; /* User-specified pointers */ int *addrlen; int flags; /* Return values */ int NumberOfBytesRead; int status; /* cannot be stored in overlapped */ } To my knowledge, there are only these two types of Asynchronous requests. Thus, using a "struct async_ops" like "struct object_ops" in the server code would probably be too much; the type field could instead be used for distinction between the two request types and the "generic" code in scheduler/synchro.c could incorporate respective case distinctions. To my experience this runs faster than code with function pointers if we have only two cases. Actually, we might even do without the "async_handler" field and decide which function to call only according to the "type" field, but that's a cosmetic issue. Unless anybody objects, I'll soon submit a patch incorporating this approach. Comments welcome, Martin -- Martin Wilck Phone: +49 5251 8 15113 Fujitsu Siemens Computers Fax: +49 5251 8 20409 Heinz-Nixdorf-Ring 1 mailto:Martin.Wilck(a)Fujitsu-Siemens.com D-33106 Paderborn http://www.fujitsu-siemens.com/primergy
Hello, I`m new to this list and I have some questions: 1.How many APIs are there in Windows? I heard something around 12000, is that correct? 2.How many of them have been implemented yet(not necessaryly bug-free)? Finally I want to add something about being able to run M$ software, specifically M$-Office. Many see a problem in there because M$ is likely to complicate things by using obscure APIs and other nasty tricks to make it hard for Office to work under Wine. This way we will be facing an uphill battle all the time. But I think we shouldn't worry too much about it. In fact, I think there are replacements for most M$ software, like Staroffice and the like. The main concern should be the ability to run the huge amounts of 3rd party software. And this will not be a problem, since the producers of this software use the well documented APIs which will always have to be publicly available. Once we run all those apps, more and more users will use Linux. As for the M$ Office, people can start using Staroffice, since it can read M$ Office fileformats... Just my 0.05$ Roland
Martin Wilck <Martin.Wilck(a)fujitsu-siemens.com> writes:
2. Unlike File I/O, The WSAOVERLAPPED struct is *not* accessed in any way, except for the hEvent field. The other fields do not carry useful information and are, according to the Winsock2 specs, reserved to service providers, i.e. we are not allowed to write to them.
But in our case Wine would be considered the service provider, so we can use the structure. It seems you should be able to use the existing code by providing a different async_handler function, and storing the needed information in the WSAOVERLAPPED structure. That would be cleaner than having the low-level file I/O routines know about winsock structures. -- Alexandre Julliard julliard(a)winehq.com
On 7 Jan 2002, Alexandre Julliard wrote:
But in our case Wine would be considered the service provider, so we can use the structure.
I wondered about that ... A "service provider" in Winsock terminology is something that implements low-level protocols below the winsock layer. To my understanding the winsock layer itself may not use it (the OS TCP/IP stack could :-). However, this may be a misunderstanding or overinterpretation on my part.
It seems you should be able to use the existing code by providing a different async_handler function, and storing the needed information in the WSAOVERLAPPED structure. That would be cleaner than having the low-level file I/O routines know about winsock structures.
Do you _really_ consider it "clean" to use the WSAOVERLAPPED fields for arbitrary, unrelated Wine-internal data? The code would have to be full of comments like /* Note: overlapped->OffsetHigh is the user-supplied struct sockaddr pointer */ I do not intend to make the low-level file I/O functions know anything about winsock. In the worst case i would have something like this in synchro.c (Pseudocode): struct async_private *ovl; if (ovl->type & ASYNC_TYPE_WINSOCK) { struct async_winsock *aws = (struct async_winsock*) ovl; aws->completion_func ( [4 arguments] ); } else { struct async_fileio *af = (struct async_fileio*) ovl; af->completion_func ( [3 arguments] ) } The only way to avoid these (admittedly unelegant) constructs is to use yet more function pointers, like ovl->call_completion_func (ovl); where ovl->call_completion_func would be a function that "knew" the number of arguments to pass to te completion function. As I said in the last mail, since we are talking about only 2 alternatives, the use of function pointers may be more elegant, but a little exaggerated to my taste. Martin -- Martin Wilck <mwilck(a)freenet.de> Physicist & Linux system engineer at FSC <Martin.Wilck(a)Fujitsu-Siemens.com>
Martin Wilck <mwilck(a)freenet.de> writes:
Do you _really_ consider it "clean" to use the WSAOVERLAPPED fields for arbitrary, unrelated Wine-internal data?
Not really, but this is apparently what Windows does. And I consider it better than having kernel32 file I/O code know about winsock specifics any more than necessary.
The only way to avoid these (admittedly unelegant) constructs is to use yet more function pointers, like
ovl->call_completion_func (ovl);
where ovl->call_completion_func would be a function that "knew" the number of arguments to pass to te completion function.
Another way is to have a completion_func inside of winsock that takes 3 args and in turn calls the user-specified func that takes 4 args. This way synchro.c does not need to know about it at all.
As I said in the last mail, since we are talking about only 2 alternatives, the use of function pointers may be more elegant, but a little exaggerated to my taste.
The basic infrastructure has to be generic enough to support all kinds of async I/O. If it cannot it should be improved, but we must not work around the lack of generality by adding winsock-specific hacks in the kernel layers. -- Alexandre Julliard julliard(a)winehq.com
On 7 Jan 2002, Alexandre Julliard wrote:
Do you _really_ consider it "clean" to use the WSAOVERLAPPED fields for arbitrary, unrelated Wine-internal data?
Not really, but this is apparently what Windows does.
I just made a test run on Win NT. The "Internal" field is used for some status information, the "InternalHigh" field for the number of bytes received/sent. There is no indication that Windows uses the "Offset"/"OffsetHigh" fields. Thus, I think we could use Internal and InternalHigh as in File IO, but we'd better leave the Offset/OffsetHigh fields alone.
Another way is to have a completion_func inside of winsock that takes 3 args and in turn calls the user-specified func that takes 4 args. This way synchro.c does not need to know about it at all.
I don't consider that very "clean", either.
The basic infrastructure has to be generic enough to support all kinds of async I/O. If it cannot it should be improved, but we must not work around the lack of generality by adding winsock-specific hacks in the kernel layers.
IMO synchro.c is for handling _async requests_. Currently it has a lot of information on ReadFileEx()-type async requests built in which is unnecessary to make async requests work and makes implementing async socket IO a lot harder than a true "generic" interface would be. There is nothing wrong with that, since File IO has been the only type of async request supported so far. All I'm asking for is to make this structure a little more generic so that I can implement async IO on sockets without a festival of casts and ugly hacks in the winsock code. IMO the synchro.c code ideally should know nothing about e.g. the number of arguments a completion function takes, neither for file nor socket IO. I have understood that you don't like the idea of case distinctions in synchro.c - fine. I was suggesting it primarily for performance reasons. Martin -- Martin Wilck Phone: +49 5251 8 15113 Fujitsu Siemens Computers Fax: +49 5251 8 20409 Heinz-Nixdorf-Ring 1 mailto:Martin.Wilck(a)Fujitsu-Siemens.com D-33106 Paderborn http://www.fujitsu-siemens.com/primergy
Martin Wilck <Martin.Wilck(a)fujitsu-siemens.com> writes:
There is nothing wrong with that, since File IO has been the only type of async request supported so far. All I'm asking for is to make this structure a little more generic so that I can implement async IO on sockets without a festival of casts and ugly hacks in the winsock code.
Sure, I'm all for making it more generic. What I objected to was that your idea of generic seemed to be doing something like: if (socket) do this else if (serial) do that else if (file) do this other thing which is just the opposite of generic IMO. But a truly generic interface is clearly needed. -- Alexandre Julliard julliard(a)winehq.com
On 7 Jan 2002, Alexandre Julliard wrote:
But in our case Wine would be considered the service provider, so we can use the structure. It seems you should be able to use the existing code by providing a different async_handler function, and storing the needed information in the WSAOVERLAPPED structure. That would be cleaner than having the low-level file I/O routines know about winsock structures.
I have overseen something: In Winsock routines it is allowed to pass a NULL LPWSAOVERLAPPED pointer, if a completion routine is specified (they may even both be NULL, then the operation is carried out synchronously). Thus the overlapped field obviously cannot be used by Wine for internal purposes, unless you want to allocate one if the user hasn't done so (but that would mean allocating a lot of useless memory space and causing confusion). I am not certain if the fact that Mike's server code identifies requests by their OVERLAPPED pointers may cause trouble with NULL OVERLAPPED pointers. I guess not, though, because for activating requests the first NULL pointer request will be scheduled, and the Winsock2 Specs state that "There is no way to cancel indiviual overlapped operations pending on a given socket". Martin -- Martin Wilck Phone: +49 5251 8 15113 Fujitsu Siemens Computers Fax: +49 5251 8 20409 Heinz-Nixdorf-Ring 1 mailto:Martin.Wilck(a)Fujitsu-Siemens.com D-33106 Paderborn http://www.fujitsu-siemens.com/primergy
participants (4)
-
Alexandre Julliard -
Martin Wilck -
Martin Wilck -
Roland