November 2001 - Wine-Devel - list.winehq.org

Re: [RFC]: Overlapped I/O implementation
by Mike McCormack Nov. 14, 2001

Nov. 14, 2001

Hi Martin, >However, I assume you have had good reasons to abaondon the old >server-centric implementation, so I'm reluctant to go that way. Not really. The main reason i did it was because the server is slow and clunky and i thought i'd be able to squeeze a bit of extra performance out of the code by putting it in the client. The best starting point for solving the problems you describe is to revert my patch that put async I/O in the client. i'll even do it if you like... Now who said that premature optimization is the root of all evil? :-) Mike ------------------------------------------ mailto:Mike_McCormack(a)start.com.au ph +82 16 430 0425 __________________________________________________________________ Get your free Australian email account at http://www.Looksmart.com.au

1 0

Yet another async I/O issue: block devices
by Martin Wilck Nov. 14, 2001

Nov. 14, 2001

Hi, I just discovered yet another problem that arises for overlapped I/O on normal disk files, more correctly, for any file on a block device. Obviously IO requests on block devices on Linux are _always blocking_. I wrote a very simple test program for reading from a floppy disk and found exactly this behaviour, no matter if O_NONBLOCK was set or not. "man 2 open" also says that O_NONBLOCK may or may not have any effect on non-socket fd's and for block devices it clearly has none. I don't know the reason for this, but it may have to do with Linux' caching algorithms. This means that - with ReadFile() etc. on regular files, the asynch code will never be executed. Instead, the call will block when trying to read immediately, and then return success. - I am not 100% sure in what context the asynchronous handlers are executed, but the read() write() calls there will be blocking as well. I assume this is not what we want. - I guess that for programs that "know" some IO is going to take a lot of time (read()s on floppies, network file systems, ...) will potentially be broken if they try to do overlapped IO and "do something else" while the IO is executing. I do see the need, therefore, to have an approach that mimics Windows' behaviour here. - The only simple approach that I can see for making this happen is creating a separate thread for reading/writing asynchronous IO. The simplest approach would actually be to do a clone() for each async request with CLONE_FILES | CLONE_VM | CLONE_PARENT. I don't know how this would fit into Wine's general synchronization approach - please give me hints. - The separate thread itself would be very simple for files on block devices, because it can simply execute the blocking read and finish when the read finishes, leaving some status information somewhere. - For other devices that _do_ support O_NONBLOCK, it may be cool to mimic this simple behaviour by clone()ing too, dup() ing the fd, resetting the O_NONBLOCK flag, and doing a blocking IO request just the same. That means to basically leave it up to the kernel to handle synchronization. This would also mean that we wouldn't have to worry about differences between files and sockets/pipes, because I'm 99% sure the kernel would do it right. I don't understand the concept of the wine service thread well enough yet to judge if that could be used for doing what I just suggested. Perhaps someone can help me out with that, I'll be studying the sources in the meantime. Cheers, Martin PS. I guess another possible approach would be using raw I/O, but I am very certain this is not what we want. -- Martin Wilck Phone: +49 5251 8 15113 Fujitsu Siemens Computers Fax: +49 5251 8 20409 Heinz-Nixdorf-Ring 1 mailto:Martin.Wilck(a)Fujitsu-Siemens.com D-33106 Paderborn http://www.fujitsu-siemens.com/primergy

2 1

Local USER classes act as global
by Joshua Thielen Nov. 14, 2001

Nov. 14, 2001

Hello, When I checked out the latest CVS, all the dialog boxes in Finale stopped working (again) . This happened since Alexandre fixed a function call from GetModuleHandleA("USER") to GetModuleHandle16("USER") in my previous GetClassInfo16 patch (thanks Alexnadre!). My patch had worked because GetModuleHandleA returned hInstance 0 for USER and Finale would proceed to register a window class with hInstance 0. Since classes with hInst=0 are treated sort of like global classes in wine, the window class was found. After more investigating, I discovered that in Win16, any window class that is registered with a USER instance handle will be found even if the hInstance passed to the function (GetClassInfo, CreateWindow etv c.) is completely different. This happens even for classes without the CS_GLOBALCLASS style. I have some small test apps if anyone is intrigued by this. Anyway, this patch implements this undocumented bug, uh.. er.. feature in wine and gets Finale working again . Changelog: Local classes registered with USER instance handle now are now found in CLASS_FindClassByAtom. Modified Files: wine/windows/wnd16.c Josh Thielen Index: wine/windows/class.c =================================================================== RCS file: /home/wine/wine/windows/class.c,v retrieving revision 1.42 diff -u -r1.42 class.c --- wine/windows/class.c 2001/11/06 20:57:27 1.42 +++ wine/windows/class.c 2001/11/13 23:23:39 @@ -279,10 +279,14 @@ * NOTES * 980805 a local class will be found now if registred with hInst=0 * and looed up with a hInst!=0. msmoney does it (jsch) + * + * Local class registered with a USER instance handle are found as if + * they were global classes. */ static CLASS *CLASS_FindClassByAtom( ATOM atom, HINSTANCE hinstance ) { - CLASS * class, *tclass=0; + CLASS * class, *tclass = 0, *user_class = 0; + HINSTANCE hUser = GetModuleHandle16("USER"); TRACE("0x%08x 0x%08x\n", atom, hinstance); @@ -293,16 +297,20 @@ if (class->style & CS_GLOBALCLASS) continue; if (class->atomName == atom) { - if (hinstance==class->hInstance || hinstance==0xffff ) + if (hinstance==class->hInstance || hinstance==0xffff) { TRACE("-- found local %p\n", class); return class; } - if (class->hInstance==0) tclass = class; + if (class->hInstance == 0) tclass = class; + else if(class->hInstance == hUser) + { + user_class = class; + } } } - /* Then search global classes */ + /* Then search global classes */ for (class = firstClass; (class); class = class->next) { @@ -312,6 +320,13 @@ TRACE("-- found global %p\n", class); return class; } + } + + /* Check if there was a local class registered with USER */ + if( user_class ) + { + TRACE("--found local USER class %p\n", user_class); + return user_class; } /* Then check if there was a local class with hInst=0*/

1 0

weird linux/solaris problem.
by Roger Fujii Nov. 13, 2001

Nov. 13, 2001

Well, I finally figured out the major problem I've been having with wine and virtualdub under solaris, but I'm not sure what to do about it. In memory/virtual.c, there are 2 problems: 1) in function anon_mmap_aligned, there is a section that tries to release extra memory. The problem is that under solaris, munmap length parameter is defined as: The munmap() function removes the mappings for pages in the range [addr, addr + len), rounding the len argument up to the next multiple of the page size as returned by sysconf(3C). It's not exactly clear what the linux munmap does, but I think that the linux one does what you think it SHOULD do, and the solaris one does what is documented (i.e. will unmap more pages than what the linux one thinks it would since it rounds UP the len). Now, I can see 2 fixes: 1 is to #ifdef __svr4 and compute the compensated len before unmapping. The other is to just not do this release (or at least not do it for __svr4). Any comments? 2) For some reason I don't quite understand, a call is made to VirtualProtect with a READONLY protection, and soon after, something tries to memcpy into that block. I'm not sure if this is a thread sync issue, or maybe this is supposed to work like c's const (ie, it's readonly AFTER the first write). Kind of puzzling, since it does work under linux. Any suggestions? (my current fix is to treat READONLY as RW, which works, but I know is wrong). Thanks. -r Roger Fujii <rmf(a)lookhere.com> Underemployed, and trying to keep it that way....

1 0

Re: [RFC]: Overlapped I/O implementation
by Martin Wilck Nov. 13, 2001

Nov. 13, 2001

Hi Mike, all, I have thought some more. Having understood the problem with the possibly duplicate process fds, it is clear that in order to treat multiple simultaneous requests correctly, there must be some server-side support on a per-IO-object basis. With the current implementation as a guideline, I see only one possibility to do this: The server maintains two semaphores for async reading and writing for each I/O object (file, socket, comm port...) that has the OVERLAPPED flag set. The semaphores have a max count of 1 for FIFOS and some larger number for random-access files (this will be set by object-type-dependent functions). The async_private struct has a Semaphore field. When the client schedules an async request, it gets the suitable semaphore handler (read/write) of the I/O object from the server and stores it in the async_private struct. The async_handler function does not start actual I/O before acquiring the semaphore, and releases it when finished. The handler is in IO_PENDING state until it gets the semaphore and remains in that state after obtining access until it releases the semaphore. This provides a rather elegant framework for limiting the number of concurrent readers and writers. I am not sure, though, if the async_handler functions are allowed to wait for semaphores ??? Perhaps we'd need to use Unix semaphores ? Any other approach I've thought of so far seems to imply a lot of client-server interaction and mangling in the NtCurrentTeb()->pending_list. Please notify me if you have better ideas. The other possible solution would be to go back to the server-centric implementation of 8 months ago, or something similar. Most likely the NtCureentTeb()->pending_list would have to be dropped in that case, and server-side communications used instead. However, I assume you have had good reasons to abaondon the old server-centric implementation, so I'm reluctant to go that way. There is another concern, though. You wrote: > When the thread waits upon any server object, all (unix) fds with > pending overlapped operations are polled on. This is a minor problem > in the design; if the thread never waits upon an object the overlapped > can never complete, however the thread must wait upon the overlapped's > hEvent to check whether it has completed... AFAICS, there will not even be an I/O request unless the client waits on something (at least in ReadFileEx()). I wonder if this can cause the advantages of async IO to be lost if a a long time elapses before the client waits on anything. Definitely, if the first wait the client issues is on the async request itself, the handler will have done nothing in the meantime and there is no real asynchronicity any more. Meanwhile all arriving data must be buffered by the kernel. This is probably no problem on a serial port, but what if you're dealing with a fully loaded ethernet connection or a local socket with a lot of traffic? Anyway, That's probably a minor issue. Please tell me if you think the semaphore idea is feasible. Martin -- Martin Wilck Phone: +49 5251 8 15113 Fujitsu Siemens Computers Fax: +49 5251 8 20409 Heinz-Nixdorf-Ring 1 mailto:Martin.Wilck(a)Fujitsu-Siemens.com D-33106 Paderborn http://www.fujitsu-siemens.com/primergy

1 0

Pajama Sam audio problems mysteriously gone; Film at eleven
by Ori Pessach Nov. 13, 2001

Nov. 13, 2001

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, In my attempts to figure out which change caused some games not to detect the sound card on my machine, I started checking out historic versions from CVS, building them, and testing whether they work or not. They all worked. All in all, I tested versions from 10/07/2001, the last version I knew worked, to the current CVS version, in a binary search pattern. The games still exhibit the problems I reported earlier - the sound is breaking up, and the colorful cursors don't show up, but at least I have something to debug now. If anybody has any ideas about the sound (waveOut*) that weren't discussed yet, I'd love to hear about them. I'd like to thank everyone for their time, especially Eric, who waded through megabytes of relay messages on what turned out to be a wild goose chase. Ori Pessach -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: http://www.oripessach.com iD8DBQE78JUUzA/uWWyw6uQRAutEAKCs/nb8dqkcoqOxKfonO8B5d5HyUgCgwImR rwT2dIFiZ5Yo7MzODLio2hw= =9RtW -----END PGP SIGNATURE----- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com

1 0

Re: Regression errors
by Gerard Patel Nov. 13, 2001

Nov. 13, 2001

At 11:06 AM 10/11/2001 -0500, you wrote: > >Thanks Gerard. I will try it Tuesday. Going out of town till then. Well, I have advanced a bit this weekend. I noticed that one of my test app does indeed show this regression after the patch we are talking about. As the case is a lot more simple than the one I was trying to debug before for this problem, I have been able to understand better what is happening. The problem shows itself in the following case : - invalidating a window - hiding it - deleting it and after that the paint count is > 0. I can make the paint count change correctly with the following change : --- winpos.c.orig Thu Nov 8 01:57:34 2001 +++ winpos.c Mon Nov 12 09:47:08 2001 @@ -883,10 +883,10 @@ if (!(winpos->flags & SWP_SHOWWINDOW) && (winpos->flags & SWP_HIDEWINDOW)) { - WIN_SetStyle( winpos->hwnd, wndPtr->dwStyle & ~WS_VISIBLE ); /* clear the update region */ -// RedrawWindow( winpos->hwnd, NULL, 0, RDW_VALIDATE | RDW_NOFRAME | -// RDW_NOERASE | RDW_NOINTERNALPAINT | RDW_ALLCHILDREN ); + RedrawWindow( winpos->hwnd, NULL, 0, RDW_VALIDATE | RDW_NOFRAME | + RDW_NOERASE | RDW_NOINTERNALPAINT | RDW_ALLCHILDREN ); + WIN_SetStyle( winpos->hwnd, wndPtr->dwStyle & ~WS_VISIBLE ); } else if ((wndPtr->dwStyle & WS_VISIBLE) && !IsRectEmpty( &oldWindowRect ) && IsRectEmpty( &newWindowRect )) Now, I don't know why this RedrawWindow call was commented out :-/ Gerard

2 1

Re: Yet another async I/O issue: block devices
by Mike McCormack Nov. 13, 2001

Nov. 13, 2001

Hi Martin, Probably better to get the ordered overlappeds working on sockets and serial ports first before trying to perfect block devices. That will bring Wine's feature set inline with Win9x overlapped I/O. I've never had anybody complain that overlapped I/O on block devices doesn't work for any program. Most programs avoid using features that only work on NT... you're likely to find that only NT services use those API calls.... and they won't work on Wine anyhow, seeing we don't support services properly yet. It seems like a good idea to try to make everything perfect from the start, but an incremental approach is usually better. Mike Original message from: Martin Wilck <Martin.Wilck(a)fujitsu-siemens.com> > > >Hi, > >I just discovered yet another problem that arises for overlapped >I/O on normal disk files, more correctly, for any file on a block device. > >Obviously IO requests on block devices on Linux are _always blocking_. >I wrote a very simple test program for reading from a floppy disk and >found exactly this behaviour, no matter if O_NONBLOCK was set or not. >"man 2 open" also says that O_NONBLOCK may or may not have any effect on >non-socket fd's and for block devices it clearly has none. > ------------------------------------------ mailto:Mike_McCormack(a)start.com.au ph +82 16 430 0425 __________________________________________________________________ Get your free Australian email account at http://www.Looksmart.com.au

1 0

Re: x11ddraw fix
by Alexandre Julliard Nov. 12, 2001

Nov. 12, 2001

geschrader <geschrader(a)mediaone.net> writes: > When X11DRV_XF86DGA2_CreatePalette( ) creates the palette > used by X11DRV_DDHAL_DestroyPalette( ) and > X11DRV_DDHAL_SetPalEntries( ), the display which is used is > returned by thread_display( ). This patch makes these two functions > also use thread_display( ) ( rather than gdi_display ). This bug was > causing a BadColor (i.e. invalid colormap) error in the X server > when X11DRV_DDHAL_SetPalEntries( ) was called, followed by > an application crash. I think it would be better to create the palette on the gdi_display, this way it can be manipulated by other threads too. Could you try if this works for you? Index: dlls/x11drv/dga2.c =================================================================== RCS file: /opt/cvs-commit/wine/dlls/x11drv/dga2.c,v retrieving revision 1.5 diff -u -r1.5 dga2.c --- dlls/x11drv/dga2.c 2001/09/11 00:32:33 1.5 +++ dlls/x11drv/dga2.c 2001/11/07 20:28:34 @@ -174,8 +174,8 @@ static DWORD PASCAL X11DRV_XF86DGA2_CreatePalette(LPDDHAL_CREATEPALETTEDATA data) { - Display *display = thread_display(); - data->lpDDPalette->u1.dwReserved1 = TSXDGACreateColormap(display, DefaultScreen(display), dga_dev, AllocAll); + data->lpDDPalette->u1.dwReserved1 = TSXDGACreateColormap(gdi_display, DefaultScreen(gdi_display), + dga_dev, AllocAll); if (data->lpColorTable) X11DRV_DDHAL_SetPalEntries(data->lpDDPalette->u1.dwReserved1, 0, 256, data->lpColorTable); @@ -199,10 +199,9 @@ static DWORD PASCAL X11DRV_XF86DGA2_SetPalette(LPDDHAL_SETPALETTEDATA data) { - Display *display = thread_display(); if ((data->lpDDSurface == X11DRV_DD_Primary) && data->lpDDPalette && data->lpDDPalette->u1.dwReserved1) { - TSXDGAInstallColormap(display, DefaultScreen(display), data->lpDDPalette->u1.dwReserved1); + TSXDGAInstallColormap(gdi_display, DefaultScreen(gdi_display), data->lpDDPalette->u1.dwReserved1); } data->ddRVal = DD_OK; return DDHAL_DRIVER_HANDLED; -- Alexandre Julliard julliard(a)winehq.com

3 2

[RFC]: Overlapped I/O implementation
by Martin Wilck Nov. 12, 2001

Nov. 12, 2001

Hi, I'd like to summarize my thoughts about an implementation of overlapped I/O in Wine that will offer full Windows functionality. I havent't thought it through in all detail, but I have a clear enough picture to write an RFC. I regard the following points as crucial: 1 "Generic" routines like ReadFileEx() need to work with files of almost arbitrary type, i.e. normal files, pipes, sockets, comm ports, ... 2 Different file types imply different semantics for overlapped I/O. In particular: a Files that don't support positioning (i.e. FIFOs, or basically everything except regular files) need to be accessed sequentially, i.e. there may only be one reader and one writer at a time, and these must be scheduled in the same order in which they were submitted by the application. Otherwise, the data read or written will be potentially scrambled. For regular files, on the other hand, spawning several I/O requests in parallel may be beneficial. b Files on some devices (in particular, block devices on Linux) need to spawn a separate thread for the async I/O operation, because I/O on such devices is always blocking, which is undesirable. Neither the process itself nor the service thread should be blocked by a time-consuming floppy I/O operation, for example. 3 The semantics of individual asynchronous I/O requests depend on the function that was called by the application, e.g. ReadFile() or WSARecv(), which take different arguments. Such calls may be mixed by the application, although it's certainly not good programming practice to do so. Based on these arguments, I'd suggest the following: A) The generic functions need to find out what the properties of a file descriptor wrt asynch I/O are. The most general way to implement this is to have an element int (*get_async) (struct object*, struct file_async*) in the obj_ops array. For non-file objects, this element would be NULL or no_get_async(). For non-overlapped file descriptors, the function would assign NULL to the file_async* argument, and return an error, a valid pointer would be assigned to file_async* otherwise. struct file_async would contain per-file information about asynch I/O, e.g. struct file_async { struct async_private *readers; /* list of async read requests */ struct async_private *writers; /* list of async write requests */ int flags /* supports positioning? Is I/O blocking on host OS ? etc. */ int (*get_timeout) (struct object *obj, int type, int count); /* function that returns timeout for device, needed e.g. for serial comm */ } I currently see no need for other file-type dependedent functions; It should be possible to write generic routines that handle the requests correctly according to the flags variable. The Wine server API would containe a file_get_async() request that returns the file_async structure associated with the file by calling the get_async() element of obj_ops. The async_private struct would look pretty similar as it looks now, with a generic part and a part that depends on the function invoked for doing async I/O (e.g. for sockets, whether ReadFile() or WSARecv() was called on the socket fd). IMO the async_private should not contain file type-dependent information which is per-file. It should rather contain a pointer to the respective file_async struct. Most importantly, the async_private struct it carries a pointer to a function that does the actual I/O, and another function pointer that handles the termination of the request (invocation of user completion routine etc.) B) The check_async() function in synchro.c operates on file descriptors rather than async requests. Once an overlapped I/O request is issued by the application, it is queued either in the readers or writers list of the async object (which is obtained from the server). If both the readers and writers lists were NULL beforehand, the fd is inserted into the list for check_async to work on. check_async() activates the first element(s) of both the readers and writers lists (where activate means that the actual read/write call is triggered). For FIFOS, only one element would be activated; for regular files several requests up to a given limit (e.g. 4). If the file is on a device where normal I/O would be blocking, a new thread is created for each activated request to satisfy it. When the requests' I/O routine (separate thread or not) is completed, it activates an event that triggers the generic "finish" routine. The finish routine calls the finish routine of the request, cleans up, and activates a new request if there is one that was queued but not activated. If both the readers and writers queue are empty after finishing the request, the file descriptor is removed from the list of handles to check in check_async(). C) If CancelIo() is called or a file is closed, all pending IO requests are removed from the file's async readers and writers lists; potential subthreads are terminated. With this approach, the wineserver would only be needed for retreiving the file's async I/O properties. The biggest change to the wine core would be the addition of a new function to obj_ops. I am looking forward to comments. Regards, Martin PS. A minor remark: The above approach would make it unnecessary to check for the return value of xyz_get_info() to be FD_TYPE_OVERLAPPED when initiating overlapped I/O. Instead, one could simply check if the async object returned by get_async() is non-NULL. -- Martin Wilck Phone: +49 5251 8 15113 Fujitsu Siemens Computers Fax: +49 5251 8 20409 Heinz-Nixdorf-Ring 1 mailto:Martin.Wilck(a)Fujitsu-Siemens.com D-33106 Paderborn http://www.fujitsu-siemens.com/primergy

1 0