- Wine-Devel - list.winehq.org

Is W really UTF-16?
by Bill Medland Jan. 9, 2002

Jan. 9, 2002

Sorry to resurrect an old subject, and please excuse my ignorance; I am only just getting started on this "character encoding" stuff. While I was working on the DrawText functions over the past many months I started wondering about when it would fail. (I'm pedantic and such things fascinate me!). The main concern I have is how to walk a W string correctly. For example while "ellipsifying" text we will need to "move the pointer to the previous character" which is currently done by decrementing the pointer by 1. But from what I currently understand that won't work if there are surrogate pairs. I see that MSDN these days keeps talking about surrogate pairs and the fact that strings of WCHARS might include them and I started to get the impression that the inputs to the W functions were truly UTF-16. I have just been looking at the discussions from April 2000 on this subject and I get the impression that we believe it too. My understanding of UTF-16 is that a surrogate pair is quite obvious and doesn't require any context information; if you come across a 16 bit value in 0xDC00 to 0xDFFF then it must be the low surrogate and there must be a high surrogate just before it. So I presumed that CharNextW must be the function that correctly walks a W string and I tried to find out what its behaviour was for the error cases (e.g. high surrogate with no low surrogate). On finding that CharNextW didn't do anything clever, it just incremented by 1, I tested and found that CharNextW always incremented by 1. So now I am confused. 1. Does anyone know under what circumstances CharNextW isn't +1 (apart from when pointing at the terminating 0) 2. Is e.g. XP really using UTF-16 or is it actually still UCS2? 3 Have we thought about how we should handle walking along a W string (in a fashion that doesn't reduced the speed to a crawl). I guess that in the short term I am expecting some sort of macro or inline. Bill Medland

2 2

Re: Immediate IO in ReadFile / WriteFile
by Mike McCormack Jan. 9, 2002

Jan. 9, 2002

Hey Martin, Another day, another problem. :-) You can't unconditionally use overlapped operations... the code used to do that exactly, and it caused all sorts of breakages. eg. some (braindead) programs check that there is data is ready to read, then do an overlapped read, but expect it *not* to go overlapped. I don't think WaitForMultipleObjectsEx(0,NULL,FALSE,0,TRUE) then read, is the right way... I think the best solution is to let the server decide. Something like this: BOOL ReadFile_overlapped () { ovp = new async_private; while(1) { ovp->func = FILE_AsyncReadService; wine_server_call(register_async); switch(GetLastError()) { case STATUS_SUCCESS: ovp->func(ovp); break; case STATUS_PENDING: return FALSE; case STATUS_TIMEOUT: goto finish_it; default: delete ovp; return FALSE; } if(count >= bytes_to_read) break; } finish_it: delete ovp; *bytes_read = count; } This is the kind of structure i was trying to describe to you in a previous private mail... i was trying to address a different problem but it should address the one you mention too. In fact, WaitCommEvent also needs to work in this way. (Somebody on wine-users has a program that needs that behaviour now.) There's a minefield of special cases... but the essence is that the wine server instructs the client to do one of the following things: * perform the action (read/write/etc) * return from the call with success * return from the call with ERROR_PENDING * wait for an event flag (ie. not return yet) Does that make sense to you? Mike Original message from: Martin Wilck <Martin.Wilck(a)fujitsu-siemens.com> > > >Hi Mike, > >I just had the following thought on the immediate IO in ReadFile() / >WriteFile() that you implemented: > >Consider the following situation: > >1. There is no data available for reading on some fd (serial port, pipe, > socket). >2. An application calls ReadFile() in overlapped mode. > since no data is there, an overlapped request is scheduled. >3. Data becomes available before the app issues a wait request of any > kind. >4. The application starts another ReadFile() request. > Now data is there, and will be read immediately. > No async request is scheduled. >5. Eventually, more data will become available, and the async request > first scheduled will also return. > >However, this basically destroys the order of the requests. >I am unsure how Windows would behave in such a case, and it is certainly >difficult to provide a test case for this situation. It can occur, though, >and my guess is this is the wrong behaviour. > >Thus, probably before doing the immediate read it should be checked >if there are asynchronous requests already scheduled for reading, >and if yes, ReadFile() should _not_ try to read immediately but go >pending. Of course, this check would require a server call. > >The same reasoning applies for writing, of course. > >Things become even more confusing if you think of several threads >reading from the same file. > >A very simple workaround would be to go pending unconditonally for >overlapped requests. Although not optimal, this would be certain >not to corrupt data. > >What do you think? > >Martin ------------------------------------------ mailto:Mike_McCormack(a)start.com.au ph +82 16 430 0425 __________________________________________________________________ Get your free Australian email account at http://www.Looksmart.com.au

3 2

A new async patch - feedback, please :-)
by Martin Wilck Jan. 9, 2002

Jan. 9, 2002

Hi Mike, all, Unfortunately this patch has become big, because i came across so many problems today and I just needed to go forward. I am sorry for this, but it has become very late tonight, and I want people to see this stuff to stop me before it's too late (if I'm on the wrong track). It compiles without warnings, but I have not tested it - may crash wine right away, I don't know. However it should illustrate my ideas well: * Full separation of async handling and file io-specific stuff. * Do not rely on LPOVERLAPPED for status data in generic code, it may be NULL. File IO code may rely on it, here it is required to be non-NULL. * Use GetOverlappedResult() to check for immediate completion (UNTESTED!) Please give me comments telling me which parts you don't like, so that I can adjust for further development. Mike, can you tell me about those apps that were causing trouble in the past ? Where can I obtain them? Do they need special hardware (modem, mouse) for test runs ? Cheers, Martin -- Martin Wilck Phone: +49 5251 8 15113 Fujitsu Siemens Computers Fax: +49 5251 8 20409 Heinz-Nixdorf-Ring 1 mailto:Martin.Wilck(a)Fujitsu-Siemens.com D-33106 Paderborn http://www.fujitsu-siemens.com/primergy ChangeLog: server/protocol.def: cancel_async(): Define new reqest type. server/async.c: cancel_async(): Implement new reqest type. include/file.h: struct async_ops: 4 methods for async_private: get_status, set_status, get_count, call_completion. struct async_private: contains only those elements which are absolutely necessary for queueing and server interaction struct async_fileio: contains all other fields. scheduler/synchro.c: register_async(): replacement for FILE_StartAsync(). Uses STATUS_USER_APC as (pseudo-) indicator to queue a new request. Hooks new requests in NtCurrentTeb()->pending_list (formerly done in calling functions). Not used by CancelIo() anymore. Finishes requests that are not in PENDING state after scheduling. finish_async(): Assume status is already set correctly. Queue call_completion unconditionally (No completion_func check needed) check_async(): Use register_async() for rescheduling. files/file.c: GetOverlappedResult(): Return ERROR_IO_INCOMPLETE if IO still pending (MSDN docs) FILE_StartAsync(): -> scheduler/synchro.c: register_async CancelIo(): Use new cancel_async server request (I felt former StartAsync() was too overloaded). Skip requests with wrong handle. fileio_call_completion_func(): Now called unconditionally, check if completion_func is non-NULL. FILE_AsyncReadService(): type adjustments according to new API FILE_AsyncWriteService(): dito FILE_ReadFileEx(): use new async API. Call GetOverlappedResult() to check for immediate completion. FILE_WriteFileEx(): dito. Now exactly equivalent to FILE_ReadFileEx() kernel/comm.c: COMM_WaitCommEventService(): Use new async API. diff -ruX diffignore CVS/wine/dlls/kernel/comm.c MW/wine/dlls/kernel/comm.c --- CVS/wine/dlls/kernel/comm.c Tue Jan 8 18:53:41 2002 +++ MW/wine/dlls/kernel/comm.c Wed Jan 9 21:20:37 2002 @@ -1514,12 +1514,13 @@ */ static void COMM_WaitCommEventService(async_private *ovp) { - LPOVERLAPPED lpOverlapped = ovp->lpOverlapped; + async_fileio *fileio = (async_fileio*) ovp; + LPOVERLAPPED lpOverlapped = fileio->lpOverlapped; TRACE("overlapped %p\n",lpOverlapped); /* FIXME: detect other events */ - *ovp->buffer = EV_RXCHAR; + *fileio->buffer = EV_RXCHAR; lpOverlapped->Internal = STATUS_SUCCESS; } @@ -1536,7 +1537,7 @@ LPOVERLAPPED lpOverlapped) /* [in/out] for Asynchronous waiting */ { int fd,ret; - async_private *ovp; + async_fileio *ovp; if(!lpOverlapped) { @@ -1547,52 +1548,35 @@ if(NtResetEvent(lpOverlapped->hEvent,NULL)) return FALSE; - lpOverlapped->Internal = STATUS_PENDING; - lpOverlapped->InternalHigh = 0; - lpOverlapped->Offset = 0; - lpOverlapped->OffsetHigh = 0; - fd = FILE_GetUnixHandle( hFile, GENERIC_WRITE ); if(fd<0) return FALSE; - ovp = (async_private *) HeapAlloc(GetProcessHeap(), 0, sizeof (async_private)); + ovp = (async_fileio *) HeapAlloc(GetProcessHeap(), 0, sizeof (async_fileio)); if(!ovp) { close(fd); return FALSE; } + + ovp->async.ops = &fileio_async_ops; + ovp->async.handle = hFile; + ovp->async.fd = fd; + ovp->async.type = ASYNC_TYPE_WAIT; + ovp->async.func = COMM_WaitCommEventService; + ovp->lpOverlapped = lpOverlapped; - ovp->func = COMM_WaitCommEventService; ovp->buffer = (char *)lpdwEvents; - ovp->fd = fd; ovp->count = 0; ovp->completion_func = 0; - ovp->type = ASYNC_TYPE_WAIT; - ovp->handle = hFile; - - ovp->next = NtCurrentTeb()->pending_list; - ovp->prev = NULL; - if(ovp->next) - ovp->next->prev=ovp; - NtCurrentTeb()->pending_list = ovp; - /* start an ASYNCHRONOUS WaitCommEvent */ - SERVER_START_REQ( register_async ) - { - req->handle = hFile; - req->overlapped = lpOverlapped; - req->type = ASYNC_TYPE_WAIT; - req->count = 0; - req->func = check_async_list; - req->status = STATUS_PENDING; - - ret=wine_server_call_err(req); - } - SERVER_END_REQ; + lpOverlapped->Internal = STATUS_USER_APC; + lpOverlapped->InternalHigh = 0; + lpOverlapped->Offset = 0; + lpOverlapped->OffsetHigh = 0; - if (!ret) - SetLastError(ERROR_IO_PENDING); + if (register_async (&ovp->async)) + SetLastError(ERROR_IO_PENDING); return FALSE; } diff -ruX diffignore CVS/wine/files/file.c MW/wine/files/file.c --- CVS/wine/files/file.c Tue Jan 8 18:57:47 2002 +++ MW/wine/files/file.c Wed Jan 9 20:55:09 2002 @@ -60,6 +60,20 @@ static HANDLE dos_handles[DOS_TABLE_SIZE]; +/* Async operations struct (see file.h) */ + +static DWORD fileio_get_async_status (async_private *ovp); +static DWORD fileio_get_async_count (async_private *ovp); +static void fileio_set_async_status (async_private *ovp, DWORD status); +static void CALLBACK fileio_call_completion_func (ULONG_PTR data); + +async_ops fileio_async_ops = +{ + fileio_get_async_status, /* get_status */ + fileio_set_async_status, /* set_status */ + fileio_get_async_count, /* get_count */ + fileio_call_completion_func /* call_completion */ +}; /*********************************************************************** * FILE_ConvertOFMode @@ -1244,35 +1258,13 @@ if(lpTransferred) *lpTransferred = lpOverlapped->InternalHigh; - SetLastError(lpOverlapped->Internal); + /* This is what the function should return accirding to MSDN specs */ + if ( lpOverlapped->Internal == STATUS_PENDING ) + SetLastError (ERROR_IO_INCOMPLETE); return (r==WAIT_OBJECT_0); } - -/*********************************************************************** - * FILE_StartAsync (INTERNAL) - * - * type==ASYNC_TYPE_NONE means cancel the indicated overlapped operation - * lpOverlapped==NULL means all overlappeds match - */ -BOOL FILE_StartAsync(HANDLE hFile, LPOVERLAPPED lpOverlapped, DWORD type, DWORD count, DWORD status) -{ - BOOL ret; - SERVER_START_REQ(register_async) - { - req->handle = hFile; - req->overlapped = lpOverlapped; - req->type = type; - req->count = count; - req->func = check_async_list; - req->status = status; - ret = wine_server_call( req ); - } - SERVER_END_REQ; - return !ret; -} - /*********************************************************************** * CancelIo (KERNEL32.@) */ @@ -1282,21 +1274,60 @@ TRACE("handle = %x\n",handle); - ovp = NtCurrentTeb()->pending_list; - while(ovp) + + for (ovp = NtCurrentTeb()->pending_list; ovp; ovp = t) { t = ovp->next; - if(FILE_StartAsync(handle, ovp->lpOverlapped, ovp->type, 0, STATUS_CANCELLED)) + if (ovp->handle != handle) continue; + + SERVER_START_REQ ( cancel_async ) { - TRACE("overlapped = %p\n",ovp->lpOverlapped); - finish_async(ovp, STATUS_CANCELLED); + req->handle = handle; + req->type = ovp->type; + req->overlapped = ovp; + wine_server_call ( req ); + if (reply->cancelled) + { + TRACE ("cancelling request: %p\n", ovp); + ovp->ops->set_status ( ovp, STATUS_CANCELLED ); + finish_async (ovp); + } } - ovp = t; + SERVER_END_REQ; } WaitForMultipleObjectsEx(0,NULL,FALSE,1,TRUE); return TRUE; } +static DWORD fileio_get_async_status (async_private *ovp) +{ + return ((async_fileio*) ovp)->lpOverlapped->Internal; +} + +static void fileio_set_async_status (async_private *ovp, DWORD status) +{ + ((async_fileio*) ovp)->lpOverlapped->Internal = status; +} + +static DWORD fileio_get_async_count (async_private *ovp) +{ + async_fileio *fileio = (async_fileio*) ovp; + DWORD ret = fileio->count - fileio->lpOverlapped->InternalHigh; + return (ret < 0 ? 0 : ret); +} + +static void CALLBACK fileio_call_completion_func (ULONG_PTR data) +{ + async_fileio *ovp = (async_fileio*) data; + TRACE ("data: %p\n", ovp); + + if (ovp->completion_func) + ovp->completion_func(ovp->lpOverlapped->Internal, + ovp->lpOverlapped->InternalHigh, + ovp->lpOverlapped); + HeapFree(GetProcessHeap(), 0, ovp); +} + /*********************************************************************** * FILE_AsyncReadService (INTERNAL) * @@ -1305,18 +1336,19 @@ */ static void FILE_AsyncReadService(async_private *ovp) { - LPOVERLAPPED lpOverlapped = ovp->lpOverlapped; + async_fileio *fileio = (async_fileio*) ovp; + LPOVERLAPPED lpOverlapped = fileio->lpOverlapped; int result, r; int already = lpOverlapped->InternalHigh; - TRACE("%p %p\n", lpOverlapped, ovp->buffer ); + TRACE("%p %p\n", lpOverlapped, fileio->buffer ); /* check to see if the data is ready (non-blocking) */ - result = pread (ovp->fd, &ovp->buffer[already], ovp->count - already, + result = pread (ovp->fd, &fileio->buffer[already], fileio->count - already, OVERLAPPED_OFFSET (lpOverlapped) + already); if ((result < 0) && (errno == ESPIPE)) - result = read (ovp->fd, &ovp->buffer[already], ovp->count - already); + result = read (ovp->fd, &fileio->buffer[already], fileio->count - already); if ( (result<0) && ((errno == EAGAIN) || (errno == EINTR))) { @@ -1334,9 +1366,9 @@ } lpOverlapped->InternalHigh += result; - TRACE("read %d more bytes %ld/%d so far\n",result,lpOverlapped->InternalHigh,ovp->count); + TRACE("read %d more bytes %ld/%d so far\n",result,lpOverlapped->InternalHigh,fileio->count); - if(lpOverlapped->InternalHigh < ovp->count) + if(lpOverlapped->InternalHigh < fileio->count) r = STATUS_PENDING; else r = STATUS_SUCCESS; @@ -1352,7 +1384,7 @@ LPOVERLAPPED overlapped, LPOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine) { - async_private *ovp; + async_fileio *ovp; int fd; TRACE("file %d to buf %p num %ld %p func %p\n", @@ -1369,10 +1401,11 @@ if(fd<0) { TRACE("Couldn't get FD\n"); + SetLastError( ERROR_INVALID_HANDLE ); return FALSE; } - ovp = (async_private *) HeapAlloc(GetProcessHeap(), 0, sizeof (async_private)); + ovp = (async_fileio *) HeapAlloc(GetProcessHeap(), 0, sizeof (async_fileio)); if(!ovp) { TRACE("HeapAlloc Failed\n"); @@ -1380,30 +1413,23 @@ close(fd); return FALSE; } + + ovp->async.ops = &fileio_async_ops; + ovp->async.handle = hFile; + ovp->async.fd = fd; + ovp->async.type = ASYNC_TYPE_READ; + ovp->async.func = FILE_AsyncReadService; + ovp->lpOverlapped = overlapped; - ovp->count = bytesToRead; ovp->completion_func = lpCompletionRoutine; - ovp->func = FILE_AsyncReadService; ovp->buffer = buffer; - ovp->fd = fd; - ovp->type = ASYNC_TYPE_READ; - ovp->handle = hFile; - - /* hook this overlap into the pending async operation list */ - ovp->next = NtCurrentTeb()->pending_list; - ovp->prev = NULL; - if(ovp->next) - ovp->next->prev = ovp; - NtCurrentTeb()->pending_list = ovp; + ovp->count = bytesToRead; - if ( !FILE_StartAsync(hFile, overlapped, ASYNC_TYPE_READ, bytesToRead, STATUS_PENDING) ) - { - /* FIXME: remove async_private and release memory */ - ERR("FILE_StartAsync failed\n"); - return FALSE; - } + /* Tell register_async that this is a new request */ + overlapped->Internal = STATUS_USER_APC; - return TRUE; + /* The request is hooked into the pending list by register_async() */ + return register_async (&ovp->async); } /*********************************************************************** @@ -1413,7 +1439,6 @@ LPOVERLAPPED overlapped, LPOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine) { - overlapped->Internal = STATUS_PENDING; overlapped->InternalHigh = 0; return FILE_ReadFileEx(hFile,buffer,bytesToRead,overlapped,lpCompletionRoutine); } @@ -1470,41 +1495,21 @@ return FALSE; } - /* see if we can read some data already (this shouldn't block) */ - result = pread( unix_handle, buffer, bytesToRead, OVERLAPPED_OFFSET(overlapped) ); - if ((result < 0) && (errno == ESPIPE)) - result = read( unix_handle, buffer, bytesToRead ); close(unix_handle); - - if(result<0) - { - if( (errno!=EAGAIN) && (errno!=EINTR) && - ((errno != EFAULT) || IsBadWritePtr( buffer, bytesToRead )) ) - { - FILE_SetDosError(); - return FALSE; - } - else - result = 0; - } + overlapped->InternalHigh = 0; - /* if we read enough to keep the app happy, then return now */ - if(result>=bytesToRead) - { - *bytesRead = result; - return TRUE; - } + if(!FILE_ReadFileEx(hFile, buffer, bytesToRead, overlapped, FILE_OverlappedComplete)) + return FALSE; - /* at last resort, do an overlapped read */ - overlapped->Internal = STATUS_PENDING; - overlapped->InternalHigh = result; - if(!FILE_ReadFileEx(hFile, buffer, bytesToRead, overlapped, FILE_OverlappedComplete)) + if ( !GetOverlappedResult (hFile, overlapped, bytesRead, FALSE) ) + { + if ( GetLastError() == ERROR_IO_INCOMPLETE ) + SetLastError ( ERROR_IO_PENDING ); return FALSE; + } - /* fail on return, with ERROR_IO_PENDING */ - SetLastError(ERROR_IO_PENDING); - return FALSE; + return TRUE; } else if ( FD_TYPE (type) == FD_TYPE_CONSOLE ) @@ -1554,18 +1559,19 @@ */ static void FILE_AsyncWriteService(struct async_private *ovp) { - LPOVERLAPPED lpOverlapped = ovp->lpOverlapped; + async_fileio *fileio = (async_fileio *) ovp; + LPOVERLAPPED lpOverlapped = fileio->lpOverlapped; int result, r; int already = lpOverlapped->InternalHigh; - TRACE("(%p %p)\n",lpOverlapped,ovp->buffer); + TRACE("(%p %p)\n",lpOverlapped,fileio->buffer); /* write some data (non-blocking) */ - result = pwrite(ovp->fd, &ovp->buffer[already], ovp->count - already, + result = pwrite(ovp->fd, &fileio->buffer[already], fileio->count - already, OVERLAPPED_OFFSET (lpOverlapped) + already); if ((result < 0) && (errno == ESPIPE)) - result = write(ovp->fd, &ovp->buffer[already], ovp->count - already); + result = write(ovp->fd, &fileio->buffer[already], fileio->count - already); if ( (result<0) && ((errno == EAGAIN) || (errno == EINTR))) { @@ -1582,9 +1588,9 @@ lpOverlapped->InternalHigh += result; - TRACE("wrote %d more bytes %ld/%d so far\n",result,lpOverlapped->InternalHigh,ovp->count); + TRACE("wrote %d more bytes %ld/%d so far\n",result,lpOverlapped->InternalHigh,fileio->count); - if(lpOverlapped->InternalHigh < ovp->count) + if(lpOverlapped->InternalHigh < fileio->count) r = STATUS_PENDING; else r = STATUS_SUCCESS; @@ -1600,7 +1606,8 @@ LPOVERLAPPED overlapped, LPOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine) { - async_private *ovp; + async_fileio *ovp; + int fd; TRACE("file %d to buf %p num %ld %p func %p stub\n", hFile, buffer, bytesToWrite, overlapped, lpCompletionRoutine); @@ -1611,45 +1618,39 @@ return FALSE; } - overlapped->Internal = STATUS_PENDING; - overlapped->InternalHigh = 0; - - if (!FILE_StartAsync(hFile, overlapped, ASYNC_TYPE_WRITE, bytesToWrite, STATUS_PENDING )) + fd = FILE_GetUnixHandle( hFile, GENERIC_WRITE ); + if(fd<0) { - TRACE("FILE_StartAsync failed\n"); + TRACE("Couldn't get FD\n"); + SetLastError( ERROR_INVALID_HANDLE ); return FALSE; } - ovp = (async_private*) HeapAlloc(GetProcessHeap(), 0, sizeof (async_private)); + ovp = (async_fileio*) HeapAlloc(GetProcessHeap(), 0, sizeof (async_fileio)); if(!ovp) { TRACE("HeapAlloc Failed\n"); SetLastError(ERROR_NOT_ENOUGH_MEMORY); + close(fd); return FALSE; } + + ovp->async.ops = &fileio_async_ops; + ovp->async.handle = hFile; + ovp->async.fd = FILE_GetUnixHandle( hFile, GENERIC_WRITE ); + ovp->async.type = ASYNC_TYPE_WRITE; + ovp->async.func = FILE_AsyncWriteService; + ovp->lpOverlapped = overlapped; - ovp->func = FILE_AsyncWriteService; ovp->buffer = (LPVOID) buffer; - ovp->count = bytesToWrite; ovp->completion_func = lpCompletionRoutine; - ovp->fd = FILE_GetUnixHandle( hFile, GENERIC_WRITE ); - ovp->type = ASYNC_TYPE_WRITE; - ovp->handle = hFile; - - if(ovp->fd <0) - { - HeapFree(GetProcessHeap(), 0, ovp); - return FALSE; - } + ovp->count = bytesToWrite; - /* hook this overlap into the pending async operation list */ - ovp->next = NtCurrentTeb()->pending_list; - ovp->prev = NULL; - if(ovp->next) - ovp->next->prev = ovp; - NtCurrentTeb()->pending_list = ovp; + /* Tell register_async that this is a new request */ + overlapped->Internal = STATUS_USER_APC; - return TRUE; + /* The request is hooked into the pending list by register_async() */ + return register_async (&ovp->async); } /*********************************************************************** @@ -1659,7 +1660,6 @@ LPOVERLAPPED overlapped, LPOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine) { - overlapped->Internal = STATUS_PENDING; overlapped->InternalHigh = 0; return FILE_WriteFileEx(hFile, buffer, bytesToWrite, overlapped, lpCompletionRoutine); @@ -1693,44 +1693,20 @@ return FALSE; } - /* see if we can write some data already (this shouldn't block) */ - - result = pwrite( unix_handle, buffer, bytesToWrite, OVERLAPPED_OFFSET (overlapped) ); - if ((result < 0) && (errno == ESPIPE)) - result = write( unix_handle, buffer, bytesToWrite ); - close(unix_handle); - - if(result<0) - { - if( (errno!=EAGAIN) && (errno!=EINTR) && - ((errno != EFAULT) || IsBadReadPtr( buffer, bytesToWrite )) ) - { - FILE_SetDosError(); - return FALSE; - } - else - result = 0; - } - - /* if we wrote enough to keep the app happy, then return now */ - if(result>=bytesToWrite) - { - *bytesWritten = result; - return TRUE; - } - - /* at last resort, do an overlapped read */ - overlapped->Internal = STATUS_PENDING; - overlapped->InternalHigh = result; + overlapped->InternalHigh = 0; - if(!FILE_WriteFileEx(hFile, buffer, bytesToWrite, overlapped, FILE_OverlappedComplete)) + if (!FILE_WriteFileEx(hFile, buffer, bytesToWrite, overlapped, FILE_OverlappedComplete)) return FALSE; + + if ( !GetOverlappedResult (hFile, overlapped, bytesWritten, FALSE) ) + { + if ( GetLastError() == ERROR_IO_INCOMPLETE ) + SetLastError ( ERROR_IO_PENDING ); + return FALSE; + } - /* fail on return, with ERROR_IO_PENDING */ - SetLastError(ERROR_IO_PENDING); - return FALSE; - + return TRUE; } else if ( FD_TYPE (type) == FD_TYPE_CONSOLE ) diff -ruX diffignore CVS/wine/include/file.h MW/wine/include/file.h --- CVS/wine/include/file.h Wed Jan 9 21:27:50 2002 +++ MW/wine/include/file.h Wed Jan 9 20:25:05 2002 @@ -33,23 +33,46 @@ /* overlapped private structure */ struct async_private; -typedef void (*async_handler)(struct async_private *ovp); + +typedef void (*async_handler) (struct async_private *ovp); +typedef void CALLBACK (*async_call_completion_func) (ULONG_PTR data); +typedef DWORD (*async_get_status) (struct async_private *ovp); +typedef DWORD (*async_get_count) (struct async_private *ovp); +typedef void (*async_set_status) (struct async_private *ovp, DWORD status); + +typedef struct async_ops +{ + async_get_status get_status; + async_set_status set_status; + async_get_count get_count; + async_call_completion_func call_completion; +} async_ops; + +extern async_ops fileio_async_ops; + typedef struct async_private { - LPOVERLAPPED lpOverlapped; - HANDLE handle; - int fd; - char *buffer; - async_handler func; - int count; - int type; - LPOVERLAPPED_COMPLETION_ROUTINE completion_func; - struct async_private *next; - struct async_private *prev; + async_ops *ops; + HANDLE handle; + int fd; + int type; + async_handler func; + struct async_private *next; + struct async_private *prev; } async_private; -extern void WINAPI check_async_list(LPOVERLAPPED ov, DWORD status); -extern void finish_async(struct async_private *ovp, DWORD status); +typedef struct async_fileio +{ + async_private async; + LPOVERLAPPED lpOverlapped; + LPOVERLAPPED_COMPLETION_ROUTINE completion_func; + char *buffer; + int count; +} async_fileio; + +extern BOOL register_async (async_private *ovp); +extern void WINAPI check_async_list(async_private *ovp, DWORD status); +extern void finish_async(async_private *ovp); /* locale-independent case conversion */ inline static char FILE_tolower( char c ) @@ -83,7 +106,6 @@ DWORD attributes, HANDLE template, BOOL fail_read_only, UINT drive_type ); extern HANDLE FILE_CreateDevice( int client_id, DWORD access, LPSECURITY_ATTRIBUTES sa ); -extern BOOL FILE_StartAsync(HANDLE handle, LPOVERLAPPED lpOverlapped, DWORD type, DWORD count, DWORD status); extern LONG WINAPI WIN16_hread(HFILE16,SEGPTR,LONG); diff -ruX diffignore CVS/wine/scheduler/synchro.c MW/wine/scheduler/synchro.c --- CVS/wine/scheduler/synchro.c Tue Jan 8 18:53:41 2002 +++ MW/wine/scheduler/synchro.c Wed Jan 9 20:27:16 2002 @@ -36,27 +36,60 @@ } } -static void CALLBACK call_completion_routine(ULONG_PTR data) +/*********************************************************************** + * register_async (INTERNAL) + * + * Manipulate async request queues. + */ +BOOL register_async (async_private *ovp) { - async_private* ovp = (async_private*)data; + BOOL ret; + DWORD status = ovp->ops->get_status (ovp); - ovp->completion_func(ovp->lpOverlapped->Internal, - ovp->lpOverlapped->InternalHigh, - ovp->lpOverlapped); - ovp->completion_func=NULL; - HeapFree(GetProcessHeap(), 0, ovp); -} + /* STATUS_USER_APC means this is a new request */ + /* STATUS_PENDING means this request is rescheduled (IO incomplete) */ -void finish_async(async_private *ovp, DWORD status) -{ - ovp->lpOverlapped->Internal=status; + if (status == STATUS_USER_APC) + { + status = STATUS_PENDING; + ovp->ops->set_status (ovp, status); + + /* hook this overlap into the pending async operation list */ + ovp->next = NtCurrentTeb()->pending_list; + ovp->prev = NULL; + if (ovp->next) ovp->next->prev = ovp; + NtCurrentTeb()->pending_list = ovp; + } - /* call ReadFileEx/WriteFileEx's overlapped completion function */ - if(ovp->completion_func) + /* The server call will destroy all except PENDING requests */ + SERVER_START_REQ(register_async) { - QueueUserAPC(call_completion_routine,GetCurrentThread(),(ULONG_PTR)ovp); + req->handle = ovp->handle; + req->overlapped = ovp; + req->type = ovp->type; + req->count = ovp->ops->get_count (ovp); + req->func = check_async_list; + req->status = status; + ret = wine_server_call( req ); } + SERVER_END_REQ; + + if (ret) ovp->ops->set_status ( ovp, GetLastError() ); + + if ( ovp->ops->get_status (ovp) != STATUS_PENDING ) + finish_async (ovp); + return !ret; +} + +/*********************************************************************** + * finish_async (INTERNAL) + * + * Called after completion or cancellation of an async request. + * Note: The completion status must be set already. + */ +void finish_async(async_private *ovp) +{ /* remove it from the active list */ if(ovp->prev) ovp->prev->next = ovp->next; @@ -70,7 +103,9 @@ ovp->prev=NULL; close(ovp->fd); - if(!ovp->completion_func) HeapFree(GetProcessHeap(), 0, ovp); + + /* Queue completion function unconditionally */ + QueueUserAPC(ovp->ops->call_completion, GetCurrentThread(), (ULONG_PTR)ovp); } /*********************************************************************** @@ -78,30 +113,33 @@ * * Process a status event from the server. */ -void WINAPI check_async_list(LPOVERLAPPED overlapped, DWORD status) +void WINAPI check_async_list(async_private *asp, DWORD status) { - async_private *ovp; + async_private *ovp = NULL; + DWORD ovp_status; /* fprintf(stderr,"overlapped %p status %x\n",overlapped,status); */ - for(ovp = NtCurrentTeb()->pending_list; ovp; ovp = ovp->next) - if(ovp->lpOverlapped == overlapped) - break; + for( ovp = NtCurrentTeb()->pending_list; ovp && ovp != asp; ovp = ovp->next ); - if(!ovp) - return; + if(!ovp) return; if(status != STATUS_ALERTED) - ovp->lpOverlapped->Internal = status; + { + ovp_status = status; + ovp->ops->set_status (ovp, status); + } + else ovp_status = ovp->ops->get_status (ovp); - if(ovp->lpOverlapped->Internal==STATUS_PENDING) - { + if( ovp_status == STATUS_PENDING) + { ovp->func(ovp); - FILE_StartAsync(ovp->handle, ovp->lpOverlapped, ovp->type, 0, ovp->lpOverlapped->Internal); - } + register_async (ovp); + ovp_status = ovp->ops->get_status (ovp); + } - if(ovp->lpOverlapped->Internal!=STATUS_PENDING) - finish_async(ovp,ovp->lpOverlapped->Internal); + if( ovp_status != STATUS_PENDING ) + finish_async(ovp); } diff -ruX diffignore CVS/wine/server/async.c MW/wine/server/async.c --- CVS/wine/server/async.c Wed Jan 2 12:50:04 2002 +++ MW/wine/server/async.c Wed Jan 9 20:04:53 2002 @@ -177,3 +177,28 @@ release_object(obj); } +DECL_HANDLER(cancel_async) +{ + struct object *obj; + + if (!(obj = get_handle_obj( current->process, req->handle, 0, NULL)) ) + return; + + if(obj->ops->queue_async) + { + struct async_queue *q = obj->ops->queue_async(obj, NULL, req->type, 0); + struct async *async; + + async = find_async(q, current, req->overlapped); + if (async) + { + destroy_async(async); + reply->cancelled = 1; + set_select_events(obj,obj->ops->get_poll_events(obj)); + } + else reply->cancelled = 0; + } + + release_object(obj); +} + diff -ruX diffignore CVS/wine/server/protocol.def MW/wine/server/protocol.def --- CVS/wine/server/protocol.def Tue Jan 8 18:58:28 2002 +++ MW/wine/server/protocol.def Wed Jan 9 19:52:33 2002 @@ -1535,7 +1535,7 @@ #define SERIALINFO_SET_ERROR 0x04 -/* Create/Destroy an async I/O */ +/* Create / reschedule an async I/O */ @REQ(register_async) handle_t handle; /* handle to comm port, socket or file */ void* func; @@ -1549,6 +1549,14 @@ #define ASYNC_TYPE_WRITE 0x02 #define ASYNC_TYPE_WAIT 0x03 +/* Cancel an async I/O */ +(a)REQ(cancel_async) + handle_t handle; + int type; + void* overlapped; +(a)REPLY + int cancelled; +(a)END /* Create a named pipe */ @REQ(create_named_pipe)

1 0

Re: We *really* need a development model change !
by Andriy Palamarchuk Jan. 9, 2002

Jan. 9, 2002

--- Alexandre Julliard <julliard(a)winehq.com> wrote: > Andriy Palamarchuk <apa3a(a)yahoo.com> writes: > > But I think implementing it now would actually make > your life easier. > This way you wouldn't have to require some magic > call to detect you > are running on Wine, you could simply rely on the > Wine makefiles to > pass you the right option. I don't see any value in information about the first TODO check in a test script. I, personally, prefer using "magic call". One more plus of autodetection - I can run the same Windows test binary on Windows and on Wine without any troubles. But I'd like to use your idea in other place. I feel uncomfortable when test does not show anything. It is not clear whether it runs any checks, whether my last changes were executed. I suggest to have switch which turns off output. By default the test will print some statistics on successful run - number of tests running, number of TODO tests, time of execution - anything else? Make script can run test with output turned off. Andriy Palamarchuk __________________________________________________ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/

1 0

Re: We *really* need a development model change !
by Andriy Palamarchuk Jan. 9, 2002

Jan. 9, 2002

Alexandre Julliard wrote: > I don't really care about the output. My point is that this output is > not necessary, a simple "assertion foo failed at line xx" on failure > and no output at all on success would work just as well. I've nothing > against such an output either, but I don't think it justifies > introducing the complexity of reusing the Test stuff. I already created small library with api like in Test::Simple - functions ok, TODO, SKIP. The implementation can be easy changed to the behaviour you described. I also suggest for library to fail on successful TODO test. TODO tests will be conditional - they are TODO only under Wine, not on Windows. Careful usage SKIP sections can easy cover differences between Windows platforms, using TODO - between different non-Microsoft Win32 implementations. On other hand, SKIP sections were important for Test::Harness, which needed to control number of tests. For your goals conditional execution of the tests is sufficient. Which approaches can you suggest to reach goals I believe are important: 1) bundling - are we going to have separate distributions - Wine with tests, tests only, Wine only? There are a lot of cases where only one of them is required. 2) development of the unit tests under Windows. Obviously, we don't need to have Wine itself when we work with unit tests on Windows. Plus, we need to create development environment, usable by Windows developers. 3) Organization of the unit tests in such way, so they can be used by other Win32 implementation projects. Conditional TODOs I suggested above will help to manage different TODO lists for different projects. 4) besides narrow-scope regression testing you suggested we need to run all the tests from time to time, even if it takes more than 1 hour to run the whole suite. The reasons for this are: a) we need to run the tests on all Windows platforms to make sure the tests correctly document their behaviour b) Wine should be reguraly tested by the whole testing suite. Some regressions will be caught in unit tests of the modules other than changed. In other cases changed module may not have tests at all, but will brake modules, dependent on it. Thanks, Andriy Palamarchuk __________________________________________________ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/

2 5

Listview ctrl
by Dimitrie O. Paun Jan. 9, 2002

Jan. 9, 2002

Hi folks, While I looking for something _small_ to do, I discovered that listview.c had two of those HEAP_strdupXXX calls that Alexandre hates so much. I figured that's got to be a 2 minute job, but I was wrong -- it turns out that the listview control needed to get Unicode enabled. So, after much work and no testing, here is a first try at doing that. The good news is that it seems it compiles. The bad news is that I have no idea how to test it. So any suggestions on how to test the listview control, or any help with the testing is greatly appreciated, as well as any comments on the patch itself. Alexandre, why is it that you want so badly to get rid of the HEAP_strdup functions? I mean, in certain cases they are not needed, and they rather signal that the code needs fixing, but in other places they _are_ needed and they convey a heck of a lot better what we're trying to do then the strange WideCharXXX function calls. -- Dimi

3 10

Re: lpOverlapped = NULL
by Mike McCormack Jan. 9, 2002

Jan. 9, 2002

Hi Martin, Sure, feel free to change it. I used the lpOverlapped to identify the async operation, but any pointer will do. The only reason for not using async_private* was to have some way to verify that the pointer was valid... Mike Original message from: Martin Wilck <Martin.Wilck(a)fujitsu-siemens.com> > > >Mike, > >I should have been aware of this before :-/ > >As mentioned in my other mail, >** Winsock allows NULL pointers for lpOverlapped. ** > >AFAICS, Mike, your server-side code uses the overlapped field >only for identifying a certain request. This will obviously >break if the overlapped field may be NULL. > >Question: Would it hurt to use the async_private pointer of a >request rather than the overlapped pointer to pass to the >server? If no, I guess that would be the way to go, because >the async_private structs are allocated by us and therefore necessarily >different and non-NULL. > >Wrt to the status and "bytes read", I already have a pretty elegant >solution which keeps the overlapped struct outside the generic code >(scheduler and server). > >Please comment, >Martin ------------------------------------------ mailto:Mike_McCormack(a)start.com.au ph +82 16 430 0425 __________________________________________________________________ Get your free Australian email account at http://www.Looksmart.com.au

1 0

lpOverlapped = NULL
by Martin Wilck Jan. 9, 2002

Jan. 9, 2002

Mike, I should have been aware of this before :-/ As mentioned in my other mail, ** Winsock allows NULL pointers for lpOverlapped. ** AFAICS, Mike, your server-side code uses the overlapped field only for identifying a certain request. This will obviously break if the overlapped field may be NULL. Question: Would it hurt to use the async_private pointer of a request rather than the overlapped pointer to pass to the server? If no, I guess that would be the way to go, because the async_private structs are allocated by us and therefore necessarily different and non-NULL. Wrt to the status and "bytes read", I already have a pretty elegant solution which keeps the overlapped struct outside the generic code (scheduler and server). Please comment, Martin -- Martin Wilck Phone: +49 5251 8 15113 Fujitsu Siemens Computers Fax: +49 5251 8 20409 Heinz-Nixdorf-Ring 1 mailto:Martin.Wilck(a)Fujitsu-Siemens.com D-33106 Paderborn http://www.fujitsu-siemens.com/primergy

1 0

struct async_private needs generalization for Winsock
by Martin Wilck Jan. 9, 2002

Jan. 9, 2002

Hi Mike, all, as I pointed out before, we must change the async_private type such that it accomodates the different needs of normal (e.g. ReadFileEx()) and Winsock (e.g. WSARecv()) async requests. The "generic" functions in scheduler/synchro.c must be able to handle these structures without knowing what type of async_private structure they have been passed. I try to summarize the differences between the two request types here: 1. Winsock async requests have a WSAOVERLAPPED struct rather than OVERLAPPED. Unlike File I/O, the routines complete if any positive number of bytes is read (there's no such thing as BytesToRead). 2. Unlike File I/O, The WSAOVERLAPPED struct is *not* accessed in any way, except for the hEvent field. The other fields do not carry useful information and are, according to the Winsock2 specs, reserved to service providers, i.e. we are not allowed to write to them. 3. Winsock functions take a scatter/gather array of type WSABUF* instead of a single buffer, and a number of WSABUFs instead of the number of bytes to read. 4. The completion routine takes an additional DWORD argument. 5. In the case of WSARecvFrom()/WSASendTo(), the socket address and length parameters passed by the user must be stored and filled in at completion. [These are the differences I've come across so far, there may be more.] I'd therefore recommend the following, similar to the treatment of "struct object" and derived types in the server code: struct async_private; typedef void (*async_handler)(struct async_private *ovp); struct async_private { unsigned int type; /* file/socket, read/write/... */ HANDLE handle; int fd; struct async_private *prev; struct async_private *next; async_handler func; } struct async_fileio { struct async_private async; char *buffer; int count; LPOVERLAPPED lpOverlapped; LPOVERLAPPED_COMPLETION_ROUTINE completion_func; } struct async_winsock { struct async_private async; LPWSABUF wsabuf; int n_wsabufs; LPWSAOVERLAPPED overlapped; LPWSAOVERLAPPED_COMPLETION_ROUTINE completion_func; struct WS_sockaddr* addr; /* User-specified pointers */ int *addrlen; int flags; /* Return values */ int NumberOfBytesRead; int status; /* cannot be stored in overlapped */ } To my knowledge, there are only these two types of Asynchronous requests. Thus, using a "struct async_ops" like "struct object_ops" in the server code would probably be too much; the type field could instead be used for distinction between the two request types and the "generic" code in scheduler/synchro.c could incorporate respective case distinctions. To my experience this runs faster than code with function pointers if we have only two cases. Actually, we might even do without the "async_handler" field and decide which function to call only according to the "type" field, but that's a cosmetic issue. Unless anybody objects, I'll soon submit a patch incorporating this approach. Comments welcome, Martin -- Martin Wilck Phone: +49 5251 8 15113 Fujitsu Siemens Computers Fax: +49 5251 8 20409 Heinz-Nixdorf-Ring 1 mailto:Martin.Wilck(a)Fujitsu-Siemens.com D-33106 Paderborn http://www.fujitsu-siemens.com/primergy

4 7

Re: We *really* need a development model change !
by Gerard Patel Jan. 9, 2002

Jan. 9, 2002

At 04:44 PM 08/01/2002 -0500, you wrote: <snip> >(Another Pythoner, cool :) ) This has given me an idea - while I don't expect it to be used in Wine, I will try to write my own test progs with it : use the *windows* python interpreter under Wine. >From the doc, it's possible to call any win32 api from it using a 'calldll' interface. If it works well, I won't need a 'test framework' - just use the standard, available tool. Gerard

4 4