January 2024 - Wine-Devel - list.winehq.org

Re: x86 SGDT emulation for Wine

by H. Peter Anvin

On January 3, 2024 7:19:02 AM PST, Sean Christopherson <seanjc(a)google.com> wrote: >On Tue, Jan 02, 2024, Elizabeth Figura wrote: >> On Wednesday, December 27, 2023 5:58:19 PM CST H. Peter Anvin wrote: >> > On December 27, 2023 2:20:37 PM PST, Elizabeth Figura <zfigura(a)codeweavers.com> wrote: >> > >Hello all, >> > > >> > >There is a Windows 98 program, a game called Nuclear Strike, which wants to >> > >do some amount of direct VGA access. Part of this is port I/O, which >> > >naturally throws SIGILL that we can trivially catch and emulate in Wine. >> > >The other part is direct access to the video memory at 0xa0000, which in >> > >general isn't a problem to catch and virtualize as well. >> > > >> > >However, this program is a bit creative about how it accesses that memory; >> > >instead of just writing to 0xa0000 directly, it looks up a segment >> > >descriptor whose base is at 0xa0000 and then uses the %es override to >> > >write bytes. In pseudo-C, what it does is: > >... > >> > A prctl() to set the UMIP-emulated return values or disable it (giving >> > SIGILL) would be easy enough. >> > >> > For the non-UMIP case, and probably for a lot of other corner cases like >> > relying on certain magic selector values and what not, the best option >> > really would be to wrap the code in a lightweight KVM container. I do *not* >> > mean running the Qemu user space part of KVM; instead have Wine interface >> > with /dev/kvm directly. >> > >> > Non-KVM-capable hardware is basically historic at this point. >> >> Sorry for the late response—I've been trying to do research on what would be >> necessary to use KVM (plus I made the poor choice of sending this during the >> holiday season...) >> >> I'm concerned that KVM is going to be difficult or even intractable. Here are >> some of the problems that I (perhaps incorrectly) understand: >> >> * As I am led to understand, there can only be one hypervisor on the machine >> at a time, > >No. Only one instance of KVM-the-module is allowed, but there is no arbitrary >limit on the number of VMs that userspace can create. The only meaningful >limitation is memory, and while struct kvm isn't tiny, it's not _that_ big. > >> and KVM has a hard limit on the number of vCPUs. >> >> The obvious way to use KVM for Wine is to make each (guest) thread a vCPU. >> That will, at the very least, run into the thread limit. In order to avoid >> that we'd need to ship a whole scheduler, which is concerning. That's a huge >> component to ship and a huge burden to keep updated. It also means we need to >> hoist *all* of the ipc and sync code into the guest, which will take an >> enormous amount of work. >> >> Moreover, because there can only be one hypervisor, and Wine is a multi- >> process beast, that means that we suddenly need to throw every process into >> the same VM. > >As above, this is wildly inaccurate. The only KVM restriction with respect to >processes is that a VM is bound to the process (address space) that created the >VM. There are no restrictions on the number of VMs that can be created, e.g. a >single process can create multiple VMs. > >> That has unfortunate implications regarding isolation (it's been a dream for >> years that we'd be able to share a single wine "VM" between multiple users), >> it complicates memory management (though perhaps not terribly?). And it means >> you can only have one Wine VM at a time, and can't use Wine at the same time >> as a "real" VM, neither of which are restrictions that currently exist. >> >> And it's not even like we can refactor—we'd have to rewrite tons of code to >> work inside a VM, but also keep the old code around for the cases where we >> don't have a VM and want to delegate scheduling to the host OS. >> >> * Besides scheduling, we need to exit the VM every time we would normally call >> into Unix code, which in practice is every time that the application does an >> NT syscall, or uses a library which we delegate to the host (including e.g. >> GPU, multimedia, audio...) > >Maybe I misinterpreted Peter's suggestion, but at least in my mind I wasn't thinking >that the entire Wine process would run in a VM, but rather Wine would run just >the "problematic" code in a VM. > Yes, the idea would be that you would run the "problematic" code inside a VM *mapped 1:1 with the external address space*, i.e. use KVM simply as a special execution mode to give you more control of the fine grained machine state like the GDT. The code that you don't want executed in the VM context simply leave unmapped in the VM page tables and set up #PF to always exit the VM context.

Jan. 4, 2024

Re: Force feedback auto center implementation?

by Tyson Whitehead

> There are tests in dinput/tests that emulate a virtual HID PID device > with various custom HID device descriptors. It can be a bit tedious and > difficult to figure, but if you can find an actual HID PID device which > has dinput auto-center support on Windows, using its descriptor could be > a good starting point to figure the required HID features. Some people > have also been successfully using Wireshark to monitor HID reports over > USB with HID PID devices to figure the expected report sequences. I bought an old MS Sidewinder FF2 off of EBay. It is a USB HID PID with device managed effect ids. I tried it on an old Windows laptop with USBPcap installed. It has auto-centering when plugged in. When you select it in MS's old fedit.exe program, it says it disables it, and it does. It also lets you run any of the effects. I captured sessions with USBPcap, but I think stuff must be being dropped, as what I get makes no sense - unplugged device, stick move freely - plug in device, usb initializes, no HID items, power light comes on and auto-centering (stick no longer moves freely) - select it in fedit.exe, usb hid pid 0x7f feature get report (parameter block pools report), auto-centering goes off (stick moves freely) (linux does this exact same request, but auto-centering remains in effect with it) - add sine effect and play it, usb hid pid 0xab (create new effect parameter block) feature set report, usb hid pid 0x98 (effect parameter block load report) feature get report (reports id 1, while linux gives id 2, which seems to agree with auto-centering being off), effect plays on stick, but no command ever sent to disable auto-centering, set the effect type-specific parameter block, or actually start the effect playback that I can see - close program, stick resumes auto-centering, with no further hid reports - repeating gives the same packets Here is a copy of the capture file from the above sessions if you want to see. I waited at least 10s between each action to clearly separate them. That gives you the USB HID reports for the Sidewinder FF2 too at time 38.536156 https://staff.sharcnet.ca/tyson/hidff3.cap > Maybe doing something similar in dinput is the way to go if that's truly > how it works, although that is only going to work with device-managed > effect ids (which I don't think we do with SDL / evdev backends). That seems like the thing to do to me too. Thanks! Tyson

Jan. 3, 2024

Re: x86 SGDT emulation for Wine

by Stefan Dösinger

Am Dienstag, 2. Januar 2024, 22:53:26 EAT schrieb Elizabeth Figura: > I'm concerned that this will be very expensive. Most VM users don't need > to exit on every syscall. While I haven't tested KVM, I think some other > Wine developers actually did a similar experiment using a hypervisor to > solve some other problem (related to 32-bit support on Mac OS), and exiting > the hypervisor was prohibitively slow. Just to add to this point, Ken Thomases and I experimented with this on Mac OS, and as Zeb said, we found it to be unworkably slow. In the d3d games we tested the performance of hypervisor + lots of exits was approximately the same as running all 32 bit guest code inside qemu's software CPU emulation, or about 5% of the performance of using native 32 bit mac processes (when they still existed). From what we could tell the cost was imposed by the CPU and not MacOS' very lightweight hypervisor API. There are obviously differences between Mac and Linux, and with Wine's new syscalls we probably don't need to exit as often as my hangover wrapper DLLs did, but combined with the other reasons Zeb listed I don't think running Wine inside KVM is ever going to be realistic.

Jan. 3, 2024

Re: x86 SGDT emulation for Wine

by Elizabeth Figura

On Wednesday, December 27, 2023 5:58:19 PM CST H. Peter Anvin wrote: > On December 27, 2023 2:20:37 PM PST, Elizabeth Figura <zfigura(a)codeweavers.com> wrote: > >Hello all, > > > >There is a Windows 98 program, a game called Nuclear Strike, which wants to > >do some amount of direct VGA access. Part of this is port I/O, which > >naturally throws SIGILL that we can trivially catch and emulate in Wine. > >The other part is direct access to the video memory at 0xa0000, which in > >general isn't a problem to catch and virtualize as well. > > > >However, this program is a bit creative about how it accesses that memory; > >instead of just writing to 0xa0000 directly, it looks up a segment > >descriptor whose base is at 0xa0000 and then uses the %es override to > >write bytes. In pseudo-C, what it does is: > > > >int get_vga_selector() > >{ > > > > sgdt(&gdt_size, &gdt_ptr); > > sldt(&ldt_segment); > > ++gdt_size; > > descriptor = gdt_ptr; > > while (descriptor->base != 0xa0000) > > { > > > > ++descriptor; > > gdt_size -= sizeof(*descriptor); > > if (!gdt_size) > > > > break; > > > > } > > > > if (gdt_size) > > > > return (descriptor - gdt_ptr) << 3; > > > > descriptor = gdt_ptr[ldt_segment >> 3]->base; > > ldt_size = gdt_ptr[ldt_segment >> 3]->limit + 1; > > while (descriptor->base != 0xa0000) > > { > > > > ++descriptor; > > ldt_size -= sizeof(*descriptor); > > if (!ldt_size) > > > > break; > > > > } > > > > if (ldt_size) > > > > return (descriptor - ldt_ptr) << 3; > > > > return 0; > > > >} > > > > > >Currently we emulate IDT access. On a read fault, we execute sidt > >ourselves, check if the read address falls within the IDT, and return some > >dummy data from the exception handler if it does [1]. We can easily enough > >implement GDT access as well this way, and there is even an out-of-tree > >patch written some years ago that does this, and helps the game run. > > > >However, there are two problems that I have observed or anticipated: > > > >(1) On systems with UMIP, the kernel emulates sgdt instructions and returns > >a consistent address which we can guarantee is invalid. However, it also > >returns a size of zero. The program doesn't expect this (cf. the way the > >loop is written above) and I believe will effectively loop forever in that > >case, or until it finds the VGA selector or hits invalid memory. > > > > I see two obvious ways to fix this: either adjust the size of the fake > > > >kernel GDT, or provide a switch to stop emulating and let Wine handle it. > >The latter may very well a more sustainable option in the long term > >(although I'll admit I can't immediately come up with a reason why, other > >than "we might need to raise the size yet again".) > > > > Does anyone have opinions on this particular topic? I can look into > > > >writing a patch but I'm not sure what the best approach is. > > > >(2) On 64-bit systems without UMIP, sgdt returns a truncated address when > >in 32-bit mode. This truncated address in practice might point anywhere in > >the address space, including to valid memory. > > > > In order to fix this, we would need the kernel to guarantee that the > > GDT > > > >base points to an address whose bottom 32 bits we can guarantee are > >inaccessible. This is relatively easy to achieve ourselves by simply > >mapping those pages as noaccess, but it also means that those pages can't > >overlap something we need; we already go to pains to make sure that > >certain parts of the address space are free. Broadly anything above the 2G > >boundary *should* be okay though. Is this feasible? > > > > We could also just decide we don't care about systems without UMIP, but > > > >that seems a bit unfortunate; it's not that old of a feature. But I also > >have no idea how hard it would be to make this kind of a guarantee on the > >kernel side. > > > > This is also, theoretically, a problem for the IDT, except that on the > > > >machines I've tested, the IDT is always at 0xfffffe0000000000. That's not > >great either (it's certainly caused some weirdness and confusion when > >debugging, when we unexpectedly catch an unrelated null pointer access) but > >it seems to work in practice. > > > >--Zeb > > > >[1] https://source.winehq.org/git/wine.git/blob/HEAD:/dlls/krnl386.exe16/ > >instr.c#l702 > > A prctl() to set the UMIP-emulated return values or disable it (giving > SIGILL) would be easy enough. > > For the non-UMIP case, and probably for a lot of other corner cases like > relying on certain magic selector values and what not, the best option > really would be to wrap the code in a lightweight KVM container. I do *not* > mean running the Qemu user space part of KVM; instead have Wine interface > with /dev/kvm directly. > > Non-KVM-capable hardware is basically historic at this point. Sorry for the late response—I've been trying to do research on what would be necessary to use KVM (plus I made the poor choice of sending this during the holiday season...) I'm concerned that KVM is going to be difficult or even intractable. Here are some of the problems that I (perhaps incorrectly) understand: * As I am led to understand, there can only be one hypervisor on the machine at a time, and KVM has a hard limit on the number of vCPUs. The obvious way to use KVM for Wine is to make each (guest) thread a vCPU. That will, at the very least, run into the thread limit. In order to avoid that we'd need to ship a whole scheduler, which is concerning. That's a huge component to ship and a huge burden to keep updated. It also means we need to hoist *all* of the ipc and sync code into the guest, which will take an enormous amount of work. Moreover, because there can only be one hypervisor, and Wine is a multi- process beast, that means that we suddenly need to throw every process into the same VM. That has unfortunate implications regarding isolation (it's been a dream for years that we'd be able to share a single wine "VM" between multiple users), it complicates memory management (though perhaps not terribly?). And it means you can only have one Wine VM at a time, and can't use Wine at the same time as a "real" VM, neither of which are restrictions that currently exist. And it's not even like we can refactor—we'd have to rewrite tons of code to work inside a VM, but also keep the old code around for the cases where we don't have a VM and want to delegate scheduling to the host OS. * Besides scheduling, we need to exit the VM every time we would normally call into Unix code, which in practice is every time that the application does an NT syscall, or uses a library which we delegate to the host (including e.g. GPU, multimedia, audio...) I'm concerned that this will be very expensive. Most VM users don't need to exit on every syscall. While I haven't tested KVM, I think some other Wine developers actually did a similar experiment using a hypervisor to solve some other problem (related to 32-bit support on Mac OS), and exiting the hypervisor was prohibitively slow. Alternatively we ship *more* components to reimplement these things inside the VM (e.g. virgl/venus for GPU hardware, other virtio bits for interacting with e.g. multimedia hardware? enough of a cache to make block I/O reasonably fast, a few layers of networking code...), which looks more and more ugly. If nothing else, it's a huge hammer to fix this one problem for an application which doesn't even currently work in Wine, *and* which isn't even a problem on sufficiently new hardware (and to fix other GDT problems which are only theoretical at this point.) --Zeb

Jan. 2, 2024

Re: Force feedback auto center implementation?

by Rémi Bernon

On 1/1/24 17:38, Tyson Whitehead wrote: > Would like to get disabling force feedback auto-centering working so > that trimming will work properly with force feedback devices in flight > simulators. > > Currently the call to disable it is a stub > > static HRESULT dinput_device_set_property( IDirectInputDevice8W > *iface, const GUID *guid, > const DIPROPHEADER *header ) > { > ... > case (DWORD_PTR)DIPROP_AUTOCENTER: > { > const DIPROPDWORD *value = (const DIPROPDWORD *)header; > if (!(impl->caps.dwFlags & DIDC_FORCEFEEDBACK)) return > DIERR_UNSUPPORTED; > if ( header->dwHow != DIPH_DEVICE) return DIERR_INVALIDPARAM; > impl->vtbl->set_property( iface, > FIXME( "DIPROP_AUTOCENTER stub!\n" ); > impl->autocenter = value->dwData; > return DI_OK; > } > ... > } > > After having looked through the wine, SDL, and kernel code and read up > on the force feedback HID physical interface device (PID) usage table > reports, I am guessing this was left as a stub because it doesn't fit > in with the current design of dipnut communicating with the backend > via HID reports. The issue being that auto centering, unlike > everything else, isn't part of the PID specification, so there isn't a > predefined report that can be used for this (see table 0x0f) > > https://www.usb.org/document-library/hid-usage-tables-14 > > Although I am no wine or Windows code expert, I am guess this leaves > the following options > > - leave it disabled in the backend and emulate it via a frontend spring effect > - abuse an existing HID report to communicate it > - add a WINE specific HID report to communicate it > - add a WINE specific IOCTL to communicate it > > For the first option, I mean have the winebus backends disable it via > the kernel or an SDL call and then have the dinput layer enable > simulate it by adding a default spring effect. This then lets the > dinput frontend disable it again by removing or disabling its spring > effect. For the second one I mean something like reserve an effect ID > for specifying auto centering. > It would be nice to figure how Windows does it, and do the same. The idea is that we should be able to communicate to HID PID devices directly for when the hidraw is used. There are tests in dinput/tests that emulate a virtual HID PID device with various custom HID device descriptors. It can be a bit tedious and difficult to figure, but if you can find an actual HID PID device which has dinput auto-center support on Windows, using its descriptor could be a good starting point to figure the required HID features. Some people have also been successfully using Wireshark to monitor HID reports over USB with HID PID devices to figure the expected report sequences. Also, maybe Windows also supports some kind of fallback (or maybe it just works like that), as long as spring effects are available, in which case we should do the same. > Both of these are inspired by the fact that from the kernel code, it > seems auto centering for HID devices is an undocumented use of the > first effect ID for a spring effect > > https://github.com/torvalds/linux/blob/610a9b8f/drivers/hid/usbhid/hid-pidf… > https://github.com/torvalds/linux/blob/610a9b8f/drivers/hid/usbhid/hid-pidf… > Maybe doing something similar in dinput is the way to go if that's truly how it works, although that is only going to work with device-managed effect ids (which I don't think we do with SDL / evdev backends). The spring effect fallback is probably a good solution (even a primary solution if figuring how native works proves too difficult). Cheers, -- Rémi Bernon <rbernon(a)codeweavers.com>

Jan. 2, 2024

Force feedback auto center implementation?

by Tyson Whitehead

Would like to get disabling force feedback auto-centering working so that trimming will work properly with force feedback devices in flight simulators. Currently the call to disable it is a stub static HRESULT dinput_device_set_property( IDirectInputDevice8W *iface, const GUID *guid, const DIPROPHEADER *header ) { ... case (DWORD_PTR)DIPROP_AUTOCENTER: { const DIPROPDWORD *value = (const DIPROPDWORD *)header; if (!(impl->caps.dwFlags & DIDC_FORCEFEEDBACK)) return DIERR_UNSUPPORTED; if ( header->dwHow != DIPH_DEVICE) return DIERR_INVALIDPARAM; impl->vtbl->set_property( iface, FIXME( "DIPROP_AUTOCENTER stub!\n" ); impl->autocenter = value->dwData; return DI_OK; } ... } After having looked through the wine, SDL, and kernel code and read up on the force feedback HID physical interface device (PID) usage table reports, I am guessing this was left as a stub because it doesn't fit in with the current design of dipnut communicating with the backend via HID reports. The issue being that auto centering, unlike everything else, isn't part of the PID specification, so there isn't a predefined report that can be used for this (see table 0x0f) https://www.usb.org/document-library/hid-usage-tables-14 Although I am no wine or Windows code expert, I am guess this leaves the following options - leave it disabled in the backend and emulate it via a frontend spring effect - abuse an existing HID report to communicate it - add a WINE specific HID report to communicate it - add a WINE specific IOCTL to communicate it For the first option, I mean have the winebus backends disable it via the kernel or an SDL call and then have the dinput layer enable simulate it by adding a default spring effect. This then lets the dinput frontend disable it again by removing or disabling its spring effect. For the second one I mean something like reserve an effect ID for specifying auto centering. Both of these are inspired by the fact that from the kernel code, it seems auto centering for HID devices is an undocumented use of the first effect ID for a spring effect https://github.com/torvalds/linux/blob/610a9b8f/drivers/hid/usbhid/hid-pidf… https://github.com/torvalds/linux/blob/610a9b8f/drivers/hid/usbhid/hid-pidf… Would be interesting in knowing what sort of implementation people think would be best and acceptable for inclusion into wine? Thanks! Tyson

Jan. 1, 2024

Re: Handling stack overflow

by Dāvis Mosāns

pirmd., 2023. g. 4. dec., plkst. 03:24 — lietotājs Dāvis Mosāns (<davispuh(a)gmail.com>) rakstīja: > > [...] > Here is small example that will deadlock Wine in exactly same way > https://gist.github.com/davispuh/6c0ae7e5a10500a5b7a759b817e66214 > In Windows such program will be killed and you'll get dialog with > button to attach > debugger. > So any thoughts how this could be fixed? It seems that in virtual_setup_exception() we need better abort_thread() that would be able to kill process that has some deadlocked threads. void abort_thread( int status ) { pthread_sigmask( SIG_BLOCK, &server_block_set, NULL ); if (InterlockedDecrement( &nb_threads ) <= 0) abort_process( status ); pthread_exit_wrapper( status ); }

Jan. 1, 2024