Martin Wilck Martin.Wilck@fujitsu-siemens.com writes:
B) The check_async() function in synchro.c operates on file descriptors rather than async requests. Once an overlapped I/O request is issued by the application, it is queued either in the readers or writers list of the async object (which is obtained from the server). If both the readers and writers lists were NULL beforehand, the fd is inserted into the list for check_async to work on.
But the list has to be stored in the server, otherwise you cannot guarantee the request order across processes. I think you need to put just about everything except the actual read/write call in the server.
check_async() activates the first element(s) of both the readers and writers lists (where activate means that the actual read/write call is triggered). For FIFOS, only one element would be activated; for regular files several requests up to a given limit (e.g. 4).
If the file is on a device where normal I/O would be blocking, a new thread is created for each activated request to satisfy it.
I don't think we want to use threads at all. This has major overhead and compatibility problems. IMO normal file I/O should always be done synchronously; it may impact performance a bit when reading from floppy, but this is better than slowing down all async operations with the need to manage threads and locks.
On 12 Nov 2001, Alexandre Julliard wrote:
But the list has to be stored in the server, otherwise you cannot guarantee the request order across processes. I think you need to put just about everything except the actual read/write call in the server.
Are your talking about threads? Wrt to processes, I don't understand the argument, because the list will be indexed by async objects which are unique to each open file, and file descriptors won't be shared across processes, right? For threads I'm not sure, but if they don't share their file descriptors it's fine for the same reason, and if they do, the algorithm I proposed will ensure correct ordering. Am I overlooking something?
I don't think we want to use threads at all. This has major overhead and compatibility problems. IMO normal file I/O should always be done synchronously; it may impact performance a bit when reading from floppy, but this is better than slowing down all async operations with the need to manage threads and locks.
OK, let's forget about this in the first place. I thought that clone() calls actually have pretty low overhead, but I may be mistaken (and too Linux-centric). And I did not think through all necessary synchonization needs.
Martin
Hi Alexandre,
On 12 Nov 2001, Alexandre Julliard wrote:
But the list has to be stored in the server, otherwise you cannot guarantee the request order across processes. I think you need to put just about everything except the actual read/write call in the server.
OK, I think I understand now (at least partially). It took me some time to understand what information can be passed between server and clients, and how.
Need to think everything over.
Reagrds, Martin
Martin Wilck wrote:
I don't think we want to use threads at all. This has major overhead and compatibility problems. IMO normal file I/O should always be done synchronously; it may impact performance a bit when reading from floppy, but this is better than slowing down all async operations with the need to manage threads and locks.
OK, let's forget about this in the first place. I thought that clone() calls actually have pretty low overhead, but I may be mistaken (and too Linux-centric). And I did not think through all necessary synchonization needs.
You may want to consider using Posix aio to implement overlapped I/O, regardless of whether it's a disk file, a network connection, or whatever. glibc has a fallback implementation of aio, and vendors are starting to provide optimized, high-quality implementations. SGI and Red Hat are working hard on it.
For more info, see http://www.kegel.com/c10k.html#aio
And then there are completion ports... more on that later. - Dan
You may want to consider using Posix aio to implement overlapped I/O, regardless of whether it's a disk file, a network connection, or whatever. glibc has a fallback implementation of aio, and vendors are starting to provide optimized, high-quality implementations. SGI and Red Hat are working hard on it.
I had a look at it, but the current implementation on my system (libc 2.2.2) is limited to files that allow lseek() and therefore only suited for disk files. Moreover, it requires pthreads. I didn't feel bold enough to check whether it works with Wine's pthreads replacement (btw, aio internally uses threads for doing IO, so if threads are the wrong thing, as Alexandre argued, aio isn't right either).
Anyway, thanks for the hint, I'll stay tuned with aio.
Martin
Martin Wilck wrote:
You may want to consider using Posix aio to implement overlapped I/O, regardless of whether it's a disk file, a network connection, or whatever. glibc has a fallback implementation of aio, and vendors are starting to provide optimized, high-quality implementations. SGI and Red Hat are working hard on it.
I had a look at it, but the current implementation on my system (libc 2.2.2) is limited to files that allow lseek() and therefore only suited for disk files.
Interesting. Probably easy to fix.
Moreover, it requires pthreads. I didn't feel bold enough to check whether it works with Wine's pthreads replacement (btw, aio internally uses threads for doing IO, so if threads are the wrong thing, as Alexandre argued, aio isn't right either).
Ben's aio doesn't make much use of threads, I think. He's actively working on it (he posted an update just today, see http://marc.theaimsgroup.com/?l=linux-aio&m=100587885602310&w=2 ).
Anyway, thanks for the hint, I'll stay tuned with aio.
Great. Mark my words, aio is going to scream; after many years of languishing as a seldom-implemented standard, it's finally becoming available on many flavors of Unix.
- Dan
-
Alexandre Julliard -
Dan Kegel -
Martin Wilck