Troy Rollo wine@troy.rollo.name writes:
@@ -412,7 +412,7 @@ { int ret, cookie;
- if (count > MAXIMUM_WAIT_OBJECTS) return STATUS_INVALID_PARAMETER_1;
- if (count > MAXIMUM_WAIT_OBJECTS || count <= 0) return
STATUS_INVALID_PARAMETER_1;
This is wrong, 0 is a valid count, and shouldn't cause an infinite loop. You'll need to debug this some more.
On Thu, 26 Jun 2003 10:16, Alexandre Julliard wrote:
Troy Rollo wine@troy.rollo.name writes:
@@ -412,7 +412,7 @@ { int ret, cookie;
- if (count > MAXIMUM_WAIT_OBJECTS) return STATUS_INVALID_PARAMETER_1;
- if (count > MAXIMUM_WAIT_OBJECTS || count <= 0) return
STATUS_INVALID_PARAMETER_1;
This is wrong, 0 is a valid count, and shouldn't cause an infinite loop. You'll need to debug this some more.
I see - you're using it for Sleep. It would appear then that the solution is to change the condition to (count > MAXIMUM_WAIT_OBJECTS || (count <= 0 && !timeout)).
The problem manifests in the Borland Library function "wait()". If you call this library function under real Windows when there are no outstanding child windows, it returns with an error. Under Wine, it gets stuck in WaitForMultipleObjectsEx, although it appears it's waiting for some input from wineserver that never comes rather than looping.
I've just tested Win2K under bochs, and WaitForMultipleObjectsEx is returning an error of 0x00000057 (ERROR_INVALID_PARAMETER) for wait() without any children, and for WaitForMultipleObjectsEx(0, 0, INFINITE, QS_ALLEVENTS, 0);
With the condition as (count > MAXIMUM_WAIT_OBJECTS || (count <= 0 && !timeout)), wine's behaviour mirrors that of Win2k, at least for the WaitForMultipleObjectsEx(0, 0, INFINITE, QS_ALLEVENTS, 0) case.
On Thu, 26 Jun 2003 15:59, Troy Rollo wrote:
With the condition as (count > MAXIMUM_WAIT_OBJECTS || (count <= 0 && !timeout)), wine's behaviour mirrors that of Win2k, at least for the WaitForMultipleObjectsEx(0, 0, INFINITE, QS_ALLEVENTS, 0) case.
While I somehow got MsgWaitForMultipleObjectsEx mixed up with WaitForMultiple ObjectsEx (too many MSDN windows open), an appropriate test with WaitForMultipleObjectsEx(0, NULL, FALSE, INFINITE, FALSE) gives the same result, but...
It now occurs to me that this fix not entirely right either - or at least it doesn't avoid creating another bug, since Sleep(INFINITE) is valid, and Sleep goes through WaitForMultipleObjectsEx.
However, since WaitForMultipleObjectsEx(0, NULL, FALSE, INFINITE, FALSE) returns an error on Win2k, this use of WaitForMultipleObjectsEx to implement Sleep must be wrong.
Troy Rollo wine@troy.rollo.name writes:
It now occurs to me that this fix not entirely right either - or at least it doesn't avoid creating another bug, since Sleep(INFINITE) is valid, and Sleep goes through WaitForMultipleObjectsEx.
However, since WaitForMultipleObjectsEx(0, NULL, FALSE, INFINITE, FALSE) returns an error on Win2k, this use of WaitForMultipleObjectsEx to implement Sleep must be wrong.
You are right, our implementation of Sleep is broken. I'll fix that.
I've been looking at the various efforts over the years to speed up wineserver or eliminate it altogether (by substituting a kernel service). Certainly the current degree of reliance on a separate process that is scheduled according to the operating system's own priorities creates significant speed problems. Some of this migh be alleviated by boosting the priority of the wineserver process or by making it a real time task, but there are still some operations it appears to be performing some operations that result in a yield and hence subject it to another wait in the queue for at least one time slice.
Even if the wineserver itself could be sped up in this way, individual processes would stull be subjected to time waiting in the queue after they make a request of wineserver. The problem becomes more severe on a heavily loaded system.
The approaches suggested so far:
Shared memory
Suffers from reliability problems which may allow one process to put the system in an inconsistent state.
Kernel module
The only effort so far seemed to put way too much into the kernel, and was abandoned over two years ago.
Other possible approaches that I haven't seen directly discussed on the wine-devel list:
An exokernel using the x86 multiring capability
Not portable to non-x86 architectures.
Cross-process calls
Also referred to under other names. This mechanism would allow one process to call into another process without giving up part of its time slice. Would require modifications to the kernel's scheduler and to standard kernel data structures, hence would have to be considered "rude". It would have one advantage in that the interface differences between this mechanism and the current mechanism could be transparent.
Have I missed any?
Anyway, I'm thinking that perhaps the kernel module approach was the right basic approach, but that the particular attempt made was merely too broad. A better approach would be to define a set of kernel calls that could be used to implement all of the other stuff (and there are currently 176 types of wineserver request, so I haven't taken the time to see if I've covered them all yet) in an in-process library.
Transparency substitution for wineserver would, I guess, be achieved by having the kernel module and its supporting library implement the wineserver requests using the wineserver data structures, and perhaps having wineserver use a non-kernel version of these facilities.
The following are things I could see immediately would be part of the kernel module:
winekernel_attach_to_kernel(char const *kernid);
winekernel_object winekernel_object_create( char const *name, void const *data, size_t size);
winekernel_object winekernel_object_open( char const *name);
int winekernel_object_close( winekernel_object obj);
int winekernel_object_namesize( winekernel_object obj);
int winekernel_object_getname( winekernel_object obj, char *name, size_t bufsize); size_t winekernel_object_size( winekernel_object obj);
int winekernel_object_getdata( winekernel_object obj, void *buffer, size_t bufsize);
int winekernel_object_setdata( winekernel_object obj, void *buffer, size_t bufsize, size_t offset);
int winekernel_object_lockobject( winekernel_object obj, int flags);
int winekernel_object_unlock( winekernel_object obj);
int winekernel_object_setacl( winekernel_object obj, wineserver_acl *acl);
int winekernel_object_attach_native_file( winekernel_object obj, int fd);
int winekernel_object_get_native_file( winekernel_object obj);
int winekernel_object_list( char *namespace, char *data, int bytes, int *bytesneeded, int flags);
There would be other things that would be in the "nice to have" category too (I'm thinking specifically about path name translation being done in a way that requires less seeks through the file system).
The way I see this working is that Wine kernel objects are stored (strangely enough) in kernel memory. This effectively amounts to a shared memory approach but with the kernel module able to clean up after a misbehaving process. In the event that cleanups after a misbehaving process were to be too complex, there would still be room for a server process that does this, and the kernel could simply assign ownership of the objects from the bad process to the server process, which gets notified via another set of calls when it receives the objects. The object name would be of the form "namespace:name", so as to have:
window:0431a9c4 file:/home/me/file.dat
A process could allow the kernel to assign the name within the namespace, so that, for example:
obj = winekernel_object_create("window", windowdata, windowdata_size); winekernel_object_gername(obj, achHWND, 15);
The idea behind the "wine_attach_to_kernel" call would be to allow for the kernel to serve either multiple different users or multiple different Windows operating system types without the objects from all of them being intermingled.
Troy,
Are these calls your own creation or do they already exist in some form in the server?
Why does the "get object data" call not have an offset, but the "set object data" does? Is offset intended to allow incremental updates of the object's data?
Do you need a "get acl" function, or is the ACL going to reside in the user's space (or in the object data?)?
If you're going to allow an "attach to kernel" functionality, shouldn't it return an opaque handle that is passed to all other functions so that the same client can make calls on different instances of the kernel? This would make it easier, for instance, to write management utilities that access more than one kernel. If you want such functionality, you probably also need a way to enumerate the running kernels.
Kelly
----- Original Message ----- From: "Troy Rollo" wine@troy.rollo.name To: wine-devel@winehq.com Sent: Monday, July 07, 2003 9:27 PM Subject: Speeding up wineserver (again)
I've been looking at the various efforts over the years to speed up
wineserver
or eliminate it altogether (by substituting a kernel service). Certainly
the
current degree of reliance on a separate process that is scheduled
according
to the operating system's own priorities creates significant speed
problems.
Some of this migh be alleviated by boosting the priority of the wineserver process or by making it a real time task, but there are still some
operations
it appears to be performing some operations that result in a yield and
hence
subject it to another wait in the queue for at least one time slice.
Even if the wineserver itself could be sped up in this way, individual processes would stull be subjected to time waiting in the queue after they make a request of wineserver. The problem becomes more severe on a heavily loaded system.
The approaches suggested so far:
Shared memory
Suffers from reliability problems which may allow one process to put the system in an inconsistent state.
Kernel module
The only effort so far seemed to put way too much into the kernel, and was abandoned over two years ago.
Other possible approaches that I haven't seen directly discussed on the wine-devel list:
An exokernel using the x86 multiring capability
Not portable to non-x86 architectures.
Cross-process calls
Also referred to under other names. This mechanism would allow one process to call into another process without giving up part of its time slice. Would require modifications to the kernel's scheduler and to standard kernel data structures, hence would have to be considered "rude". It would have one advantage in that the interface differences between this mechanism and the current mechanism could be transparent.
Have I missed any?
Anyway, I'm thinking that perhaps the kernel module approach was the right basic approach, but that the particular attempt made was merely too broad.
A
better approach would be to define a set of kernel calls that could be
used
to implement all of the other stuff (and there are currently 176 types of wineserver request, so I haven't taken the time to see if I've covered
them
all yet) in an in-process library.
Transparency substitution for wineserver would, I guess, be achieved by
having
the kernel module and its supporting library implement the wineserver requests using the wineserver data structures, and perhaps having
wineserver
use a non-kernel version of these facilities.
The following are things I could see immediately would be part of the
kernel
module:
winekernel_attach_to_kernel(char const *kernid);
winekernel_object winekernel_object_create( char const *name, void const *data, size_t size);
winekernel_object winekernel_object_open( char const *name);
int winekernel_object_close( winekernel_object obj);
int winekernel_object_namesize( winekernel_object obj);
int winekernel_object_getname( winekernel_object obj, char *name, size_t bufsize);
size_t winekernel_object_size( winekernel_object obj);
int winekernel_object_getdata( winekernel_object obj, void *buffer, size_t bufsize);
int winekernel_object_setdata( winekernel_object obj, void *buffer, size_t bufsize, size_t offset);
int winekernel_object_lockobject( winekernel_object obj, int flags);
int winekernel_object_unlock( winekernel_object obj);
int winekernel_object_setacl( winekernel_object obj, wineserver_acl *acl);
int winekernel_object_attach_native_file( winekernel_object obj, int fd);
int winekernel_object_get_native_file( winekernel_object obj);
int winekernel_object_list( char *namespace, char *data, int bytes, int *bytesneeded, int flags);
There would be other things that would be in the "nice to have" category
too
(I'm thinking specifically about path name translation being done in a way that requires less seeks through the file system).
The way I see this working is that Wine kernel objects are stored
(strangely
enough) in kernel memory. This effectively amounts to a shared memory approach but with the kernel module able to clean up after a misbehaving process. In the event that cleanups after a misbehaving process were to be too complex, there would still be room for a server process that does
this,
and the kernel could simply assign ownership of the objects from the bad process to the server process, which gets notified via another set of
calls
when it receives the objects. The object name would be of the form "namespace:name", so as to have:
window:0431a9c4 file:/home/me/file.dat
A process could allow the kernel to assign the name within the namespace,
so
that, for example:
obj = winekernel_object_create("window", windowdata, windowdata_size); winekernel_object_gername(obj, achHWND, 15);
The idea behind the "wine_attach_to_kernel" call would be to allow for the kernel to serve either multiple different users or multiple different
Windows
operating system types without the objects from all of them being intermingled.
On Tue, 8 Jul 2003 12:49, Kelly Leahy wrote:
Are these calls your own creation or do they already exist in some form in the server?
They're my own creation. I'm looking more at the abstract regarding what would be necessary rather than implementation at this stage. It may be that there's a fundamental flaw I've missed, in which case I'll have to think of some entirely different approach.
Why does the "get object data" call not have an offset, but the "set object data" does? Is offset intended to allow incremental updates of the object's data?
That was the idea, and there's no particular reason why the calls aren't symmetrical other than that's the way I thought of them.
Do you need a "get acl" function, or is the ACL going to reside in the user's space (or in the object data?)?
A "get acl" may be necessary, depending on whether the user space code needs access to this information of course. One reason user space code may need access is if one process is if more than one process can modify the ACL. The ACL may be the basis for the NT security APIs that deal with ACLs, which would definitely require a "get acl".
If you're going to allow an "attach to kernel" functionality, shouldn't it return an opaque handle that is passed to all other functions so that the same client can make calls on different instances of the kernel?
Possibly. I was only thinking of the use of an ordinary client process, which only needs to be able to attach to one logical Wine kernel.
This might be an approach to implement permissions and services in a consistent way, whitout needing to give admin rights to the admin account ( per user admin ? ).
Basicly the kernel part could handle the services ( run as user <xxx> ), the question is how to handle interaction between users on the same machine ? shared folders ? network neighbourhood ?
Will give this some thought.
/ Lars Segerlund
Kelly Leahy wrote:
Troy,
Are these calls your own creation or do they already exist in some form in the server?
Why does the "get object data" call not have an offset, but the "set object data" does? Is offset intended to allow incremental updates of the object's data?
Do you need a "get acl" function, or is the ACL going to reside in the user's space (or in the object data?)?
If you're going to allow an "attach to kernel" functionality, shouldn't it return an opaque handle that is passed to all other functions so that the same client can make calls on different instances of the kernel? This would make it easier, for instance, to write management utilities that access more than one kernel. If you want such functionality, you probably also need a way to enumerate the running kernels.
Kelly
----- Original Message ----- From: "Troy Rollo" wine@troy.rollo.name To: wine-devel@winehq.com Sent: Monday, July 07, 2003 9:27 PM Subject: Speeding up wineserver (again)
I've been looking at the various efforts over the years to speed up
wineserver
or eliminate it altogether (by substituting a kernel service). Certainly
the
current degree of reliance on a separate process that is scheduled
according
to the operating system's own priorities creates significant speed
problems.
Some of this migh be alleviated by boosting the priority of the wineserver process or by making it a real time task, but there are still some
operations
it appears to be performing some operations that result in a yield and
hence
subject it to another wait in the queue for at least one time slice.
Even if the wineserver itself could be sped up in this way, individual processes would stull be subjected to time waiting in the queue after they make a request of wineserver. The problem becomes more severe on a heavily loaded system.
The approaches suggested so far:
Shared memory
Suffers from reliability problems which may allow one process to put the system in an inconsistent state.
Kernel module
The only effort so far seemed to put way too much into the kernel, and was abandoned over two years ago.
Other possible approaches that I haven't seen directly discussed on the wine-devel list:
An exokernel using the x86 multiring capability
Not portable to non-x86 architectures.
Cross-process calls
Also referred to under other names. This mechanism would allow one process to call into another process without giving up part of its time slice. Would require modifications to the kernel's scheduler and to standard kernel data structures, hence would have to be considered "rude". It would have one advantage in that the interface differences between this mechanism and the current mechanism could be transparent.
Have I missed any?
Anyway, I'm thinking that perhaps the kernel module approach was the right basic approach, but that the particular attempt made was merely too broad.
A
better approach would be to define a set of kernel calls that could be
used
to implement all of the other stuff (and there are currently 176 types of wineserver request, so I haven't taken the time to see if I've covered
them
all yet) in an in-process library.
Transparency substitution for wineserver would, I guess, be achieved by
having
the kernel module and its supporting library implement the wineserver requests using the wineserver data structures, and perhaps having
wineserver
use a non-kernel version of these facilities.
The following are things I could see immediately would be part of the
kernel
module:
winekernel_attach_to_kernel(char const *kernid);
winekernel_object winekernel_object_create( char const *name, void const *data, size_t size);
winekernel_object winekernel_object_open( char const *name);
int winekernel_object_close( winekernel_object obj);
int winekernel_object_namesize( winekernel_object obj);
int winekernel_object_getname( winekernel_object obj, char *name, size_t bufsize);
size_t winekernel_object_size( winekernel_object obj);
int winekernel_object_getdata( winekernel_object obj, void *buffer, size_t bufsize);
int winekernel_object_setdata( winekernel_object obj, void *buffer, size_t bufsize, size_t offset);
int winekernel_object_lockobject( winekernel_object obj, int flags);
int winekernel_object_unlock( winekernel_object obj);
int winekernel_object_setacl( winekernel_object obj, wineserver_acl *acl);
int winekernel_object_attach_native_file( winekernel_object obj, int fd);
int winekernel_object_get_native_file( winekernel_object obj);
int winekernel_object_list( char *namespace, char *data, int bytes, int *bytesneeded, int flags);
There would be other things that would be in the "nice to have" category
too
(I'm thinking specifically about path name translation being done in a way that requires less seeks through the file system).
The way I see this working is that Wine kernel objects are stored
(strangely
enough) in kernel memory. This effectively amounts to a shared memory approach but with the kernel module able to clean up after a misbehaving process. In the event that cleanups after a misbehaving process were to be too complex, there would still be room for a server process that does
this,
and the kernel could simply assign ownership of the objects from the bad process to the server process, which gets notified via another set of
calls
when it receives the objects. The object name would be of the form "namespace:name", so as to have:
window:0431a9c4 file:/home/me/file.dat
A process could allow the kernel to assign the name within the namespace,
so
that, for example:
obj = winekernel_object_create("window", windowdata, windowdata_size); winekernel_object_gername(obj, achHWND, 15);
The idea behind the "wine_attach_to_kernel" call would be to allow for the kernel to serve either multiple different users or multiple different
Windows
operating system types without the objects from all of them being intermingled.
On Tue, 5 Aug 2003 15:38, Lars Segerlund wrote:
[Earlier stuff describing a kernel approach deleted]
This might be an approach to implement permissions and services in a consistent way, whitout needing to give admin rights to the admin account ( per user admin ? ).
I've been thinking this too. The kernel APIs I was suggesting are really just a new IPC mechanism that is sort of like shared memory, but with better access control facilities and better potential to set up a naming structure that cooperating processes can use. It seems to me it would have applications beyond just wine. Of course this is without sacrificing the principle that it needs to have value in improving the efficiency of wine.
I'm calling this mechanism "lockboxes" for now. Unfortunately I probably won't have much time to work on it until late November.
Basicly the kernel part could handle the services ( run as user <xxx> ), the question is how to handle interaction between users on the same machine ? shared folders ? network neighbourhood ?
The principle I'm starting from is to provide mechanisms so that what amount to subsystems can be implemented in user mode without compromising security or efficiency, and trying to figure out the minimum necessary set of new kernel functionality to support this.
I'm attaching the header file that constitutes the current definition of the lockbox API.
Solaris would need no kernel modification at all if you went to the doors interface. It looks made for it.
alan
On Tue, 8 Jul 2003 13:12, Alan Hargreaves - Product Technical Support (APAC) wrote:
Solaris would need no kernel modification at all if you went to the doors interface. It looks made for it.
Interesting. Of course doors is a cross-process call mechanism, so it would also fit in with that alternative. In fact it may well be possible to implement wineserver as a door server on Solaris with a negligible amount of additional work.
There is also an implementation of something like doors for Linux http://ldoor.sourceforge.net/, although that uses the mechanism of replacing the calling process' address space with that of the door server. This would avoid the problems associated with having to modify the scheduler, but taken literally the door server doesn't have its own files, only those of the caller. There might be some separate issues created by this.
tir, 08.07.2003 kl. 04.27 skrev Troy Rollo:
I've been looking at the various efforts over the years to speed up wineserver or eliminate it altogether (by substituting a kernel service). Certainly the current degree of reliance on a separate process that is scheduled according to the operating system's own priorities creates significant speed problems. Some of this migh be alleviated by boosting the priority of the wineserver process or by making it a real time task, but there are still some operations it appears to be performing some operations that result in a yield and hence subject it to another wait in the queue for at least one time slice.
Which operations are you really concerned about? Just synchronization (WaitFor*, SetEvent, etc), file requests (get_unix_fd), or "all of them" in general?
Even if the wineserver itself could be sped up in this way, individual processes would stull be subjected to time waiting in the queue after they make a request of wineserver. The problem becomes more severe on a heavily loaded system.
Yeah, tell me about it. It seems that when trying to run some of the latest 3D games under WineX, about half of the cpu time is spent on wineserver communication, on average.
The approaches suggested so far:
Shared memory
Suffers from reliability problems which may allow one process to put the system in an inconsistent state.Kernel module
The only effort so far seemed to put way too much into the kernel, and was abandoned over two years ago.
And then there's the approach that Alexandre suggested when he saw that big kernel module. Currently I have a design in mind that would implement something along those lines, but since Gav seems in love with the shared-memory approach, I don't know if I'd get that much time to work on it.
Basically this module I'd like to write would not change too much in the existing wineserver. But instead of unix sockets (AF_UNIX), we'd make the wineserver use a "wine socket" type (AF_WINE) implemented in the module, which would work a bit like unix sockets, but which would intercept wineserver requests from wine clients and process the most time-critical types straight in the kernel. It'd pass any request it doesn't implement on to the regular wineserver. The regular wineserver could do ioctls for stuff like "signal this object" to tell the module when it should satisfy wait requests handled by the module and such. And each wineserver listening socket would be its own namespace.
While it's probably going to be hard to get all the details and races right, this would accelerate many server requests while also avoiding any kind of user-space shared memory, which has always been the main concern. And it wouldn't be trying to do too much.
On Tue, 8 Jul 2003 13:38, Ove Kaaven wrote:
Which operations are you really concerned about? Just synchronization (WaitFor*, SetEvent, etc), file requests (get_unix_fd), or "all of them" in general?
The most visible problem for me is file requests. Some of the things I use open hundreds and even thousands of files in quick succession, and this is painful. But generally, I'd say "all of it" - any yield() is required is going to hurt throughput. Which particular ones affect a particular app will reflect what the app is doing.
-
Alan Hargreaves - Product Technical Support (APAC) -
Alexandre Julliard -
Kelly Leahy -
Lars Segerlund -
Ove Kaaven -
Troy Rollo