We need ntoskrnl to run some subset of drivers on wine. Notably cd protection drivers. They are not a hardware drivers, but a means to access something that is not accessible from user space. So in a sense ntoskrnl is just a process to run those drivers in, and send/receive requests to/from those drivers. You need to keep in mind that this is a kernel level, so some things will not work, and some things have to work that normally do not in the user space. Also ntoskrnl for windows _is_ what wineserver for wine. User space programs talk to these drivers in the following way: 1. At the start up, driver creates symbolic link in "\DosDevices\something". 2. Program calls CreateFile("\\.\something",..) to get a handle to the device. 3. Uses ReadFile/WriteFile/DeviceIoControl on that handle. (For now we need only DeviceIoControl. Read/Write could be added later.) Here is a list of the most pressing problem and the way to solve them. Any ideas and comments will be greatly appreciated, Vitaliy. Problems: 1. Open handle to a device - NtCreateFile. 2. Call DeviceIoControl on a device. (This also includes (1) since we need to know that this is not a cdrom, vxd or something else.) 3. Resolve user space handle to the DEVICE_OBJECT. (Assuming that we don't call ntoskrnl through wineserver). Solutions for (1): 1.1. Use a hack to determine that passed name is a device name. (Device names don't have colon in them). 1.2. Check against all possible names other then device names. (A:..Z:, etc.) 1.3. Try it last if all else fails. (since name spaces don't intersect this should work.) 1.4. Use object manager to lookup the name to find out what is it. (Unless we open all handles using this method it will add one extra wineserver call.) Solutions for (2): ntdll->ntoskrnl 2.1. Use named pipe to communicate with ntoskrnl process. Pass commands over the pipe in internal format. 2.2. Use UNIX IPC to talk to ntoskrnl. (pipe, socket, etc.) ntdll->wineserver->ntoskrnl 2.3. Call wineserver which will call ntoskrnl using 2.1. or 2.2. 2.4. Call wineserver which will call ntoskrnl using proper IoXXX functions from ntoskrnl to create an IRP. ntoskrnl will process them and notify wineserver when call is complete. (This is the most complicated and closer represents what windows does. There are still a problem communicating to ntoskrnl. It is probably the way to go in the future. But that will require number of things implemented which are not trivial.) Solutions for (2): MSDN quote: "Note that handles are not used for accessing device objects or driver objects." 3.1. Use wineserver to resolve handle to some internal identifier and pass it to ntoskrnl. 3.2. Pass handle and calling process id (pid, global handle, etc.) to ntoskrnl. ntoskrnl will call wineserver to resolve handle to internal id. (This handle will have to be either cached, or it will have to be created for every call.) 3.3. (requires 2.3. or 2.4.) Wineserver will resolve handle and pass internal id to ntoskrnl. (Unless we want wineserver do much more than that.)
Vitaliy Margolen wrote:
We need ntoskrnl to run some subset of drivers on wine. Notably cd protection drivers. They are not a hardware drivers, but a means to access something that is not accessible from user space. So in a sense ntoskrnl is just a process to run those drivers in, and send/receive requests to/from those drivers.
You need to keep in mind that this is a kernel level, so some things will not work, and some things have to work that normally do not in the user space. Also ntoskrnl for windows _is_ what wineserver for wine.
I think that the only way to make device drivers work properly is to eliminate wineserver completely, and make it what it really wants to be, which is a user mode implementation of ntoskrnl, something like User Mode Linux. That would require being able to write to a process's memory from the ntoskrnl, which is what your really need to implement ReadFile and WriteFile so device driver can work properly. That project is really beyond the scope of Wine... but an interesting one none the less :) Mike
Vitaliy Margolen schrieb:
We need ntoskrnl to run some subset of drivers on wine. Notably cd protection drivers. They are not a hardware drivers, but a means to access something that is not accessible from user space. So in a sense ntoskrnl is just a process to run those drivers in, and send/receive requests to/from those drivers.
You need to keep in mind that this is a kernel level, so some things will not work, and some things have to work that normally do not in the user space. Also ntoskrnl for windows _is_ what wineserver for wine.
Why implement ntoskrnl as a seperate process plus inventing a new IPC protocol to talk to it? As you said: "ntoskrnl for windows _is_ what wineserver for wine.". So why not implement the needed ntoskrnl stuff into wineserver? Peter
Peter Beutner wrote:
Why implement ntoskrnl as a seperate process plus inventing a new IPC protocol to talk to it? As you said: "ntoskrnl for windows _is_ what wineserver for wine.". So why not implement the needed ntoskrnl stuff into wineserver?
Great idea, but Alexadre doesn't want drivers running in wineserver. Ivan
"Ivan" == Ivan Leo Puoti <ivanleo(a)gmail.com> writes:
Ivan> Peter Beutner wrote: >> Why implement ntoskrnl as a seperate process plus inventing a new IPC >> protocol to talk to it? As you said: "ntoskrnl for windows _is_ what >> wineserver for wine.". So why not implement the needed ntoskrnl stuff >> into wineserver? Ivan> Great idea, but Alexadre doesn't want drivers running in Ivan> wineserver. Was this discussed on wine-devel? It's a pity if design decisions get explained on the winehackers IRC channel, so there are great chances to miss it... -- Uwe Bonnes bon(a)elektron.ikp.physik.tu-darmstadt.de Institut fuer Kernphysik Schlossgartenstrasse 9 64289 Darmstadt --------- Tel. 06151 162516 -------- Fax. 06151 164321 ----------
Ivan Leo Puoti schrieb:
Peter Beutner wrote:
Why implement ntoskrnl as a seperate process plus inventing a new IPC protocol to talk to it? As you said: "ntoskrnl for windows _is_ what wineserver for wine.". So why not implement the needed ntoskrnl stuff into wineserver?
Great idea, but Alexadre doesn't want drivers running in wineserver. Any reasons given?
Ivan
Hi, On Tue, Sep 06, 2005 at 11:32:05AM +0100, Ivan Leo Puoti wrote:
Peter Beutner wrote:
Any reasons given?
Yes, he doesn't want driver in the wineserver.
Well, that reasoning cannot remain without further discussion if there truly is a need for such a feature. Either having a technically superiour isolated ntoskrnl framework is sufficiently easy and feasible and compatible, or an alternative could perhaps be to add this functionality to wineserver yet disable critical parts of it by default in order to not compromise stability. Andreas Mohr
On Tue, Sep 06, 2005 at 01:46:21PM +0200, Andreas Mohr wrote:
Hi,
On Tue, Sep 06, 2005 at 11:32:05AM +0100, Ivan Leo Puoti wrote:
Peter Beutner wrote:
Any reasons given?
Yes, he doesn't want driver in the wineserver.
Well, that reasoning cannot remain without further discussion if there truly is a need for such a feature.
Either having a technically superiour isolated ntoskrnl framework is sufficiently easy and feasible and compatible, or an alternative could perhaps be to add this functionality to wineserver yet disable critical parts of it by default in order to not compromise stability.
This ntoskrnl is meant to load binary windows drivers. It will never live in the wineserver due to this reason ;) Ciao, Marcus
On Tuesday 06 September 2005 07:38, Alexandre Julliard wrote:
Peter Beutner <p.beutner(a)gmx.net> writes:
Any reasons given?
Stability is the obvious reason.
Stability? It'd still be at most as unstable as a native windows session, so I don't see a big deal with that. If wineserver and/or user process crashes, so what? That'd very likely happen on the native platform as well, so I fail to see anything wrong with that approach from the stability perspective... Kuba
Kuba Ober wrote:
On Tuesday 06 September 2005 07:38, Alexandre Julliard wrote:
Peter Beutner <p.beutner(a)gmx.net> writes:
Any reasons given?
Stability is the obvious reason.
Stability? It'd still be at most as unstable as a native windows session, so I don't see a big deal with that. If wineserver and/or user process crashes, so what? That'd very likely happen on the native platform as well, so I fail to see anything wrong with that approach from the stability perspective...
Except that Windows doesn't have missing functions or functions that aren't quite completely implemented. As Alexandre says, there is a ton of stuff that would have to be duplicated in order to be able to load Windows drivers in the wineserver. For example, PE loader, virtual memory, exceptions / signal handling and threading. Also note that the wineserver is single threaded (and it would be a big task to make it thread-safe) and that whenever you are calling driver code then all Wine apps are blocked. -- Rob Shearman
On Tuesday 06 September 2005 10:27, you wrote:
Kuba Ober wrote:
On Tuesday 06 September 2005 07:38, Alexandre Julliard wrote:
Peter Beutner <p.beutner(a)gmx.net> writes:
Any reasons given?
Stability is the obvious reason.
Stability? It'd still be at most as unstable as a native windows session, so I don't see a big deal with that. If wineserver and/or user process crashes, so what? That'd very likely happen on the native platform as well, so I fail to see anything wrong with that approach from the stability perspective...
Except that Windows doesn't have missing functions or functions that aren't quite completely implemented.
I was looking at it entirely from the overall stability standpoint, assuming that wine will otherwise move forward. If we assume that wine will forever be stuck where it's now then of course this wouldn't make big sense. But then it's a strategic decision that will affect the long run, mostly, as right now there'd be like one or two drivers supported anyway (copy protection I'd assume)... Kuba
On Tue, 6 Sep 2005, Alexandre Julliard wrote:
Peter Beutner <p.beutner(a)gmx.net> writes:
Any reasons given?
Stability is the obvious reason. And also of course the fact that we have most of the code we need in ntdll already and none of it in wineserver.
Just trying to move things forward and understand because I'm curious: So the way to do it would be to have ntdll load the drivers in the process using them? Wouldn't that cause problems if a driver expects to be able to maintain shared information across processes? Do we know if this is going to be a problem with the drivers we are interested in? -- Francois Gouget fgouget(a)free.fr http://fgouget.free.fr/ Good judgment comes from experience, and experience comes from bad judgment -- Barry LePatner
Tuesday, September 6, 2005, 7:34:44 AM, Francois Gouget wrote:
On Tue, 6 Sep 2005, Alexandre Julliard wrote:
Peter Beutner <p.beutner(a)gmx.net> writes:
Any reasons given?
Stability is the obvious reason. And also of course the fact that we have most of the code we need in ntdll already and none of it in wineserver.
Just trying to move things forward and understand because I'm curious:
So the way to do it would be to have ntdll load the drivers in the process using them? This obviously will not work. Drivers need their own space (i.e. kernel space). Besides, that will complicate things a lot. Also that means adding to ntdll something that does not belong there. On the other hand, this allows to implement buffer method METHOD_NEITHER the way it should be (running in the process's thread context).
Wouldn't that cause problems if a driver expects to be able to maintain shared information across processes? Yes it will. Is I understand, drivers don't have a concept of a process. So all the information they maintain, is global to all processes.
Do we know if this is going to be a problem with the drivers we are interested in? It looks like all protection systems have some kind of loader, and the game itself in some kind of wrapper or encrypted. A loader can set some information up in the driver. This information will be used by the game process and/or wrapper to do some magic/decrypt game before starting it.
All this means that we need a separate process emulating kernel internals (to some extent) just enough for these drivers to work. Wineserver is not a suitable place for drivers, since it's not a "windows process". We can't load drivers as a dll to the calling process either. This leaves only one option - dedicated process for drivers. Vitaliy
Alexandre Julliard schrieb:
Peter Beutner <p.beutner(a)gmx.net> writes:
Any reasons given?
Stability is the obvious reason. And also of course the fact that we have most of the code we need in ntdll already and none of it in wineserver.
Yupp most of the code maybe already in ntdll. But imo it is quite as ugly, from a design point of view, to call ntdll functions from inside the ntoskrnl. It should be the other way around, shouldn't it?
Peter Beutner wrote:
Yupp most of the code maybe already in ntdll. But imo it is quite as ugly, from a design point of view, to call ntdll functions from inside the ntoskrnl. It should be the other way around, shouldn't it?
Sure, you just have to convince Alexandre that it's a good idea to move all the Nt* APIs to ntoskrnl, you're welcome to try but I think it's something you won't s do before wine 8.0 Ivan
Ivan Leo Puoti wrote:
Peter Beutner wrote:
Yupp most of the code maybe already in ntdll. But imo it is quite as ugly, from a design point of view, to call ntdll functions from inside the ntoskrnl. It should be the other way around, shouldn't it?
Sure, you just have to convince Alexandre that it's a good idea to move all the Nt* APIs to ntoskrnl, you're welcome to try but I think it's something you won't s do before wine 8.0
Every time you feel like saying "you just have to convince Alexandre that it's a good idea" it should be a warning sign that what you are proposing isn't really a good idea. In this case, ntdll shouldn't be calling ntoskrnl because we aren't going to emulate the NT kernel to user apps. That is the job for ReactOS. So yes, ntoskrnl should call ntdll functions because ntoskrnl is just a user-mode client like any other app/dll. Rob
Rob Shearman wrote:
Ivan Leo Puoti wrote:
Sure, you just have to convince Alexandre that it's a good idea to move all the Nt* APIs to ntoskrnl, you're welcome to try but I think it's something you won't s do before wine 8.0
Every time you feel like saying "you just have to convince Alexandre that it's a good idea" it should be a warning sign that what you are proposing isn't really a good idea.
In this case, ntdll shouldn't be calling ntoskrnl because we aren't going to emulate the NT kernel to user apps. That is the job for ReactOS. So yes, ntoskrnl should call ntdll functions because ntoskrnl is just a user-mode client like any other app/dll.
Rob
Actually I was being sarcastic, anyway moving that stuff without causing regressions would take a long time, it's something beyond the scope of wine 1.0 regardless of whether it's good or bad. Ivan.
First let me thank everyone helping me and Ivan to get ntoskrnl moving forward. Summary of all the ideas and requirements: 1. wineserver should not be used to run drivers (absence of required functionality, single threaded, stability issues) 2. ntoskrnl should be used to run drivers only and link to ntdll and not the other way around. Unless we implement int 0x2e handling and replace wineserver with ntoskrnl. (this is wine not reactos, ntdll already has all the required functionality) 3. Drivers can not be ran in a process that requires them (drivers we concerned about keep information that is shared between different processes, drivers require their own environment that is different from a user process environment, ntdll has no means nor should it have them to run a driver) And that's it. This validates original design by Ivan Leo Puoti for ntoskrnl. Now, there are still unanswered questions: 1. How to talk to ntoskrnl (directly from ntdll or through wineserver). 2. How to get information required for ntoskrnl from wineserver. 3. How to identify this is a device operation and not a file/named pipe/mail slot, etc. I know folks you have more ideas, keep them coming. Vitaliy Margolen
3. Drivers can not be ran in a process that requires them (drivers we concerned about keep information that is shared between different processes, drivers require their own environment that is different from a user process environment, ntdll has no means nor should it have them to run a driver)
Is this really true for secdrv? Ciao, Marcus
Wednesday, September 7, 2005, 11:16:49 AM, Marcus Meissner wrote:
3. Drivers can not be ran in a process that requires them (drivers we concerned about keep information that is shared between different processes, drivers require their own environment that is different from a user process environment, ntdll has no means nor should it have them to run a driver) Is this really true for secdrv? Well you might me right about this. I see no sure prove signs that game wrapper depends on secdrv driver being loaded by game loader. This is all in relation to SafeDisc 1. I have no idea if this is true for SafeDisc 2.
But I have one more game, that uses different protection system. It has 3 drivers installed. And it does depend on them "Auto start" during the system startup. I have no information about how they work. That said I think it makes more sense having ntoskrnl handle drivers because eventually that what we will end up with in the long run. Vitaliy
Marcus Meissner wrote:
Is this really true for secdrv?
Even if secdrv 1.0 doesn't need it, the day secdrv x.0 needs it you'll have to fix everything all over again. Apart from anything else drivers expect to be able to talk to each other, and apps expect to be able to interact with devices regardless of who started the driver, and that's impossible without ntoskrnl or very very very ugly hacks. Ivan.
--- Vitaliy Margolen <wine-devel(a)kievinfo.com> wrote:
First let me thank everyone helping me and Ivan to get ntoskrnl moving forward.
Summary of all the ideas and requirements: 1. wineserver should not be used to run drivers (absence of required functionality, single threaded, stability issues) 2. ntoskrnl should be used to run drivers only and link to ntdll and not the other way around. Unless we implement int 0x2e handling and replace wineserver with ntoskrnl. (this is wine not reactos, ntdll already has all the required functionality)
Using int 0x2e and other Linux-specific hacks like kernel modules, limits portability of Wine, you'll never get it working on Windows itself.
3. Drivers can not be ran in a process that requires them (drivers we concerned about keep information that is shared between different processes, drivers require their own environment that is different from a user process environment, ntdll has no means nor should it have them to run a driver)
I am worried about performance. When you change from a real driver to a separate process with IPCs, your response time goes from function calls to context switches.
And that's it.
This validates original design by Ivan Leo Puoti for ntoskrnl. Now, there are still unanswered questions: 1. How to talk to ntoskrnl (directly from ntdll or through wineserver).
You can't use plain UNIX IPC's or sockets, since when you call ReadFile(), the driver doesn't know that the other end of its socket received a read(). You need a protocol that sends the driver a message then does a read() on the socket. Either way ntdll needs to changed. For performance reasons, I would reduce the number of processes each call has to go through, so avoid going through wineserver.
2. How to get information required for ntoskrnl from wineserver. 3. How to identify this is a device operation and not a file/named pipe/mail slot, etc.
I've wanted this functionality for a while now. Look at ntdll/cdrom.c for a (really ugly) way that the current CDROM driver uses (it does a linear search through a table of HANDLE / fd structures). The only clean solution I see is to store more data in the wineserver regarding each handle.
I know folks you have more ideas, keep them coming.
I'm attaching a few possible solutions I mailed to Alexandre a few months ago (he ignored them :-( ).
Vitaliy Margolen
Bye Damjan ______________________________________________________ Click here to donate to the Hurricane Katrina relief effort. http://store.yahoo.com/redcross-donate3/ About STI STI (Still Image Interface) is the still image system used on Windows 98 onwards. WIA, found on Windows XP, uses STI and provides more functionality. So what does it do? Well, previously, image acquisition looked like so Application (eg. Imaging) | | v TWAIN frontend (part of Windows) | | v TWAIN backend (written by scanner manufacturer) | | (other software, kernel-mode drivers, etc.) | | v Scanner Note how arrows go down only. So scanning was a pull protocol, with an application requesting an image from TWAIN, and requests going all the way down to the scanner. So how do all the buttons you find on scanners nowdays work? STI allows scanning to be done via a push protocol: +----->Application (eg. Imaging) | | | | | v | TWAIN frontend (part of Windows) | | | | | v | TWAIN backend (written by vendor) | | | | | v +------------STI (written by microsoft) | ^ | | v | STI minidriver (written by vendor) | ^ User mode | | ------------------- | | Kernel mode | | v | STI port driver (written by vendor or by Microsoft) | ^ | | v | Scanner STI does several things: * An application can register itself with STI to handle scanner events (like button presses). This process is permanent (it's stored in the registry) until the application explicitly unregisters. The application can handle a particular scanner event, or all scanner events. If no application is registered, the event is ignored; if multiple applications are registered for the event, STI asks to user to pick one (and it doesn't store the picked application anywhere). The application is then started with command line parameters that indicate the device and the event. * It loads the minidrivers when a known device is plugged in. A device becomes known through the still image class installer which registers the device with STI when its drivers are installed. The minidrivers are used by the TWAIN backend (or other software) to access the scanner, and they provide notification of scanner events. Minidrivers in turn communicate with kernel-mode drivers using CreateFile(), DeviceIoControl(), WriteFile() and ReadFile(). * A still-image event monitor (STIMON.EXE on Windows 98, STISVC.EXE on Windows 2000) needs to be running to load minidrivers when new scanners are plugged in and act when scanner events occur. * Some kernel mode drivers are written by Microsoft, others have to be written by the vendor. On Windows 2000, USBSCAN.SYS, SCSISCAN.SYS and SERSCAN.SYS are there; Windows 98 doesn't have USBSCAN.SYS so vendors provide their own driver that's used by their STI minidriver. The interface to these drivers is well-documented :-) and can easily be forwarded through libusb on Linux (already been done, in tests). STI is written in COM (yes it's ugly). There are 4 classes used: IStillImage - STI.DLL creates an object of this class with the StiCreateInstance() function. Apps use this to register and unregister themselves with STI, it can be used to view the plugged-in devices, get and set registry information for a particular device, and to create instances of IStiDevice. IStiDevice - the interface through which you access a device, and register for device events. IStiUSD - minidrivers implement this, it's mostly the same as IStiDevice (without any device event handler lists). IStiDeviceControl - allows minidrivers to query the mode in which they were open and the name of their device port. Plans for Wine -------------- It would be tempting to implement all classes with generic implementations, but that's impossible. The drivers for my scanner rely on the vendor-supplied minidriver to do something special, replacing it with a generic minidriver breaks everything. Vendor minidrivers must therefore be loaded. So here comes the problem: when the minidriver tries to open USBSCAN.SYS, what then? As it is, the device path "\\.\USBSCAN.SYS" will parse as a VxD in wine, and thus fail. Since VxD's do not implement ReadFile() and WriteFile(), or the correct range of IOCTL codes, they cannot be used to implement wine's own USBSCAN.SYS. On ReactOS, on the other hand, it would be possible to do a kernel-mode USBSCAN.SYS, but on wine? It would be possible to patch (more like hack apart, really) CreateFile() and/or NtCreateFile() to deal with USBSCAN.SYS specifically by allocating a special handle, and ReadFile() WriteFile() and DeviceIoControl() then checking for that handle, and invoking special behaviour that converts the data received into libusb requests. This works - the only reason it hasn't been done is that it requires substantial changes to the wineserver, and some people on the winedevel mailing list were threatening mutiny... While in my opinion the changes would be beneficial (DeviceIoControl() at the moment only works on CD-ROMS, ha ha), I'll leave that to some other brave soul. My current line of thought is that things must be hacked far more locally to STI, ie. the minidrivers when loaded, must have their DLL imports patched to route CreateFile(), CloseHandle(), ReadFile(), WriteFile() and DeviceIoControl() through some internal functions that selectively fall back on either libusb or the real functions in KERNEL32.DLL. It is possible, just so ugly it's unbelievable, but it works... After some thought, here's what I came up with: * NtDeviceIoControl() or whatever it's called needs to be patched a bit. It should use DEVICE_TYPE_FROM_CTL_CODE() and demultiplex which function to call from that; so if it's a CDROM/DVDROM call CDROM_DeviceIoControl(), if it's a FILE_DEVICE_USB_SCAN call for example USBSCAN_DeviceIoControl(). * As for creating the handle, patch CreateFile() and/or NtCreateFile() to deal with USBSCAN.SYS in particular (proper handling requires more work, but it's a start). They should use the pipe() call to create a pipe, fork() or in some other way create a process / thread, NO! Cannot read() on a pipe() - how would the other end know when it has to usb_bulk_read(), and how many bytes to send? Must implement this properly... --- I've been working on an STI (Still Image system) implementation for wine, and I either need good ideas, or changes to some wine fundamentals (NTDLL and wineserver). I'd like your opinion. The part of STI that won't work in wine's current state, is the STI minidriver, which is a DLL, provided by the hardware vendor, that allows applications to access the still image device: Minidriver (written by hardware vendor) | User mode | ----------- | Kernel mode | | v Kernel-mode driver (written by hardware vendor / Microsoft) | v Device (I'm using USB as an example; STI works with SCSI, serial and infrared hardware too.) The minidriver communicates with the kernel-mode driver (which is named, for example, "USBSCAN.SYS") using CreateFile() then DeviceIoControl(), ReadFile() and WriteFile(). The problems are: [1] CreateFile() thinks "\\.\USBSCAN" is a VxD, and so immediately fails. Adding a VxD of this name won't work, since ReadFile() and WriteFile() are not supported by VxD's anyway. [2] DeviceIoControl(), ReadFile() and WriteFile() need to be implemented either using libusb, or Linux-specific ioctl()'s that work on the /proc filesystem (of course, hardware on other busses is a different story). At present, NtDeviceIoControlFile() calls CDROM_DeviceIoControl(), and NtReadFile() and NtWriteFile() call read() and write(), which isn't very useful. Possible solutions, in increasing order of preference: 1. GENERIC MINIDRIVER --------------------- Make a minidriver that implements the IStiDevice interface and uses libusb directly, skipping USBSCAN.SYS. Disadvantages: * Tried it, doesn't work with my scanner and about 30 others (the vendor's minidriver does something special) 2. KERNEL MODULE ---------------- Implement a kernel module that would accept read() write() and ioctl() calls, and deal with them like USBSCAN.SYS would. NtDeviceIoControlFile() gets modified to call ioctl() when IOCTL_CODE_TO_DEVICE_TYPE(dwControlCode) == DEVICE_TYPE_USB_SCAN. Advantages: * Only CreateFile() and NtDeviceIoControlFile() need to be changed Disadvantages: * completely Linux-specific (zero portability) * ioctl codes inconsistent with Linux kernel numbering (unlikely to be included in kernel tree) * cannot be written by me (no experience coding kernel modules) * problems porting to different Linux kernel versions (eg. the SANE project had a scanner kernel module; it died out with 2.4 kernels). * needs several kernel modules for USB, SCSI, serial and IR. 3. DLL PATCHING --------------- Patch DLL imports for the minidriver so CreateFile(), ReadFile(), WriteFile() and DeviceIoControl() get dynamically linked to alternative implementations that use libusb functions. Advantages: * no changes to existing wine code Disadvantages: * wine doesn't support DLL patching * if the minidriver starts doing asynchronous I/O ... * many versions of each function, for USB, SCSI, etc. * it's a hack - doesn't solve the real problem 4. TYPED HANDLES ---------------- Modify HANDLE to somehow store handle type, or else function pointers to functions used on handles, like reading, writing and ioctl's. NTDLL functions like NtWriteFile(), NtReadFile() and NtDeviceIoControlFile() should use the handle type to demultiplex the I/O request to the correct function. So for example when NtDeviceIoControlFile() is called on a handle of type (say) WINE_HANDLE_USB_SCAN, the function UsbScanDeviceIoControl() is called, when it's called on a handle of type WINE_HANDLE_CDROM then CDROM_DeviceIoControl() is called. It would help too if some generic data (eg. void*) could be associated with each handle, so code managing a handle family could store some internal data there. Personally I advise: struct HandleData { VOID (*Close)(HANDLE handle); DWORD (*Ioctl)(HANDLE handle, /* Other DeviceIoControl() parameters */); DWORD (*Read)(HANDLE handle, LPVOID *buffer, DWORD size); DWORD (*Write)(HANDLE handle, LPVOID *buffer, DWORD size); VOID *internalData; }; #define HANDLE_TO_HandleData(h) ... Advantages: * Flexible; easy to add other hardware support in general * For STI: SCSI, serial and IR easier to add * Less hacks in other wine code (have you seen "struct cdrom_hash" and the comment "This should be removed when a proper device interface is implemented" in dlls/ntdll/cdrom.c?) Disadvantages: * the winedev mailing list wasn't very thrilled last time I mentioned it ("device drivers don't belong in wine") * might require changes to the wineserver (does it?) I personally like option 4 (typed handles), but since you maintain wineserver, and know wine better than anyone, what do you think? Any other ideas?
Damjan Jovanovic wrote:
I am worried about performance. When you change from a real driver to a separate process with IPCs, your response time goes from function calls to context switches.
Actually performance isn't' bad, we can load the game splash screen at least as fast as windows does. Drivers expect to be able to talk to each other, and processes expect to be able to interact with drivers/devices regardless of which process loaded the driver, so driver must have their own process to run in. Ivan.
Damjan Jovanovic wrote:
Using int 0x2e and other Linux-specific hacks like kernel modules, limits portability of Wine, you'll never get it working on Windows itself.
I have no idea why you'd want to port Wine back to Windows to use Windows device drivers.
4. TYPED HANDLES ---------------- Modify HANDLE to somehow store handle type, or else function pointers to functions used on handles, like reading, writing and ioctl's. NTDLL functions like NtWriteFile(), NtReadFile() and NtDeviceIoControlFile() should use the handle type to demultiplex the I/O request to the correct function. So for example when NtDeviceIoControlFile() is called on a handle of type (say) WINE_HANDLE_USB_SCAN, the function UsbScanDeviceIoControl() is called, when it's called on a handle of type WINE_HANDLE_CDROM then CDROM_DeviceIoControl() is called. It would help too if some generic data (eg. void*) could be associated with each handle, so code managing a handle family could store some internal data there. Personally I advise: struct HandleData { VOID (*Close)(HANDLE handle); DWORD (*Ioctl)(HANDLE handle, /* Other DeviceIoControl() parameters */); DWORD (*Read)(HANDLE handle, LPVOID *buffer, DWORD size); DWORD (*Write)(HANDLE handle, LPVOID *buffer, DWORD size); VOID *internalData; }; #define HANDLE_TO_HandleData(h) ...
Wine used to have function call tables like this about 5 years, and typed handles about a year ago. Those things have been removed on purpose. You can't store function pointers in the wineserver, as each process may have those functions at a different address. It looks like Alexandre has made the design decision to live with a file handle mapping to a single Unix file descriptor a while back. That makes things difficult for people who want to write code to load and use device drivers in any consistent fashion. Mike
Mike McCormack <mike(a)codeweavers.com> writes:
Wine used to have function call tables like this about 5 years, and typed handles about a year ago. Those things have been removed on purpose.
You can't store function pointers in the wineserver, as each process may have those functions at a different address.
It looks like Alexandre has made the design decision to live with a file handle mapping to a single Unix file descriptor a while back. That makes things difficult for people who want to write code to load and use device drivers in any consistent fashion.
The problem is that our I/O performance sucks, and we need to improve it by having a faster mechanism to map from handle to file descriptor. That doesn't mean we cannot have a special case for drivers, but it needs to be done in a way that doesn't impact normal I/O. This means that for instance doing a system call to retrieve the handle type is not an option. There are other possible ways though, for example by having the mapping return some sort of magic file descriptor that can somehow be identified as belonging to a device (details left as an exercise for the reader ;-) Then we could imagine a generic mechanism to redirect I/O calls for specific devices to the wineserver, which could then either handle them directly or forward them to ntoskrnl for the case of a native driver. If done right, such a mechanism could probably also be used to fix the async I/O support, which is currently in pretty bad shape. -- Alexandre Julliard julliard(a)winehq.org
Alexandre Julliard wrote:
Then we could imagine a generic mechanism to redirect I/O calls for specific devices to the wineserver, which could then either handle them directly or forward them to ntoskrnl for the case of a native driver.
That requires heavy wineserver modifications because we can't do blocking calls, so that means making it multi threaded, and thread safe. Once wine has figured out it's a native thing it might as well call ntoskrnl itself. Ivan.
Ivan Leo Puoti <ivanleo(a)gmail.com> writes:
That requires heavy wineserver modifications because we can't do blocking calls, so that means making it multi threaded, and thread safe. Once wine has figured out it's a native thing it might as well call ntoskrnl itself.
I'm not sure how you reach that conclusion, there's absolutely no reason to make the server multithreaded. And calling ntoskrnl directly is not as easy as it seems. -- Alexandre Julliard julliard(a)winehq.org
Alexandre Julliard wrote:
I'm not sure how you reach that conclusion, there's absolutely no reason to make the server multithreaded. And calling ntoskrnl directly is not as easy as it seems.
I don't see how calling it indirectly is any easier, actually It seems even more complex to me, even if that's just MHO. Ivan.
--- Mike McCormack <mike(a)codeweavers.com> wrote:
Damjan Jovanovic wrote:
Using int 0x2e and other Linux-specific hacks like kernel modules, limits portability of Wine, you'll never get it working on Windows itself.
I have no idea why you'd want to port Wine back to Windows to use Windows device drivers.
What if you want to run Windows XP apps on Windows 98? I used a bad example, any non-Linux OS (FreeBSD, MacOS, ReactOS) would have the same problem.
4. TYPED HANDLES ---------------- Modify HANDLE to somehow store handle type, or else function pointers to functions used on handles, like reading, writing and ioctl's. NTDLL functions like NtWriteFile(), NtReadFile() and NtDeviceIoControlFile() should use the handle type to demultiplex the I/O request to the correct function. So for example when NtDeviceIoControlFile() is called on a handle of type (say) WINE_HANDLE_USB_SCAN, the function UsbScanDeviceIoControl() is called, when it's called on a handle of type WINE_HANDLE_CDROM then CDROM_DeviceIoControl() is called. It would help too if some generic data (eg. void*) could be associated with each handle, so code managing a handle family could store some internal data there. Personally I advise: struct HandleData { VOID (*Close)(HANDLE handle); DWORD (*Ioctl)(HANDLE handle, /* Other DeviceIoControl() parameters */); DWORD (*Read)(HANDLE handle, LPVOID *buffer, DWORD size); DWORD (*Write)(HANDLE handle, LPVOID *buffer, DWORD size); VOID *internalData; }; #define HANDLE_TO_HandleData(h) ...
Wine used to have function call tables like this about 5 years, and typed handles about a year ago. Those things have been removed on purpose.
You can't store function pointers in the wineserver, as each process may have those functions at a different address.
You could store arbitrary pointers, usable from wine itself.
It looks like Alexandre has made the design decision to live with a file handle mapping to a single Unix file descriptor a while back. That makes things difficult for people who want to write code to load and use device drivers in any consistent fashion.
Thus our problems arose.
Mike
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
participants (14)
-
Alexandre Julliard -
Andreas Mohr -
Damjan Jovanovic -
Francois Gouget -
Ivan Leo Puoti -
Kuba Ober -
Marcus Meissner -
Marcus Meissner -
Mike McCormack -
Peter Beutner -
Rob Shearman -
Robert Shearman
-
Uwe Bonnes -
Vitaliy Margolen