December 2023 - Wine-Devel - list.winehq.org

Re: x86 SGDT emulation for Wine

by H. Peter Anvin

On December 27, 2023 2:20:37 PM PST, Elizabeth Figura <zfigura(a)codeweavers.com> wrote: >Hello all, > >There is a Windows 98 program, a game called Nuclear Strike, which wants to do >some amount of direct VGA access. Part of this is port I/O, which naturally >throws SIGILL that we can trivially catch and emulate in Wine. The other part >is direct access to the video memory at 0xa0000, which in general isn't a >problem to catch and virtualize as well. > >However, this program is a bit creative about how it accesses that memory; >instead of just writing to 0xa0000 directly, it looks up a segment descriptor >whose base is at 0xa0000 and then uses the %es override to write bytes. In >pseudo-C, what it does is: > >int get_vga_selector() >{ > sgdt(&gdt_size, &gdt_ptr); > sldt(&ldt_segment); > ++gdt_size; > descriptor = gdt_ptr; > while (descriptor->base != 0xa0000) > { > ++descriptor; > gdt_size -= sizeof(*descriptor); > if (!gdt_size) > break; > } > > if (gdt_size) > return (descriptor - gdt_ptr) << 3; > > descriptor = gdt_ptr[ldt_segment >> 3]->base; > ldt_size = gdt_ptr[ldt_segment >> 3]->limit + 1; > while (descriptor->base != 0xa0000) > { > ++descriptor; > ldt_size -= sizeof(*descriptor); > if (!ldt_size) > break; > } > > if (ldt_size) > return (descriptor - ldt_ptr) << 3; > > return 0; >} > > >Currently we emulate IDT access. On a read fault, we execute sidt ourselves, >check if the read address falls within the IDT, and return some dummy data >from the exception handler if it does [1]. We can easily enough implement GDT >access as well this way, and there is even an out-of-tree patch written some >years ago that does this, and helps the game run. > >However, there are two problems that I have observed or anticipated: > >(1) On systems with UMIP, the kernel emulates sgdt instructions and returns a >consistent address which we can guarantee is invalid. However, it also returns >a size of zero. The program doesn't expect this (cf. the way the loop is >written above) and I believe will effectively loop forever in that case, or >until it finds the VGA selector or hits invalid memory. > > I see two obvious ways to fix this: either adjust the size of the fake >kernel GDT, or provide a switch to stop emulating and let Wine handle it. The >latter may very well a more sustainable option in the long term (although I'll >admit I can't immediately come up with a reason why, other than "we might need >to raise the size yet again".) > > Does anyone have opinions on this particular topic? I can look into >writing a patch but I'm not sure what the best approach is. > >(2) On 64-bit systems without UMIP, sgdt returns a truncated address when in >32-bit mode. This truncated address in practice might point anywhere in the >address space, including to valid memory. > > In order to fix this, we would need the kernel to guarantee that the GDT >base points to an address whose bottom 32 bits we can guarantee are >inaccessible. This is relatively easy to achieve ourselves by simply mapping >those pages as noaccess, but it also means that those pages can't overlap >something we need; we already go to pains to make sure that certain parts of >the address space are free. Broadly anything above the 2G boundary *should* be >okay though. Is this feasible? > > We could also just decide we don't care about systems without UMIP, but >that seems a bit unfortunate; it's not that old of a feature. But I also have >no idea how hard it would be to make this kind of a guarantee on the kernel >side. > > This is also, theoretically, a problem for the IDT, except that on the >machines I've tested, the IDT is always at 0xfffffe0000000000. That's not >great either (it's certainly caused some weirdness and confusion when >debugging, when we unexpectedly catch an unrelated null pointer access) but it >seems to work in practice. > >--Zeb > >[1] https://source.winehq.org/git/wine.git/blob/HEAD:/dlls/krnl386.exe16/ >instr.c#l702 > > A prctl() to set the UMIP-emulated return values or disable it (giving SIGILL) would be easy enough. For the non-UMIP case, and probably for a lot of other corner cases like relying on certain magic selector values and what not, the best option really would be to wrap the code in a lightweight KVM container. I do *not* mean running the Qemu user space part of KVM; instead have Wine interface with /dev/kvm directly. Non-KVM-capable hardware is basically historic at this point.

Dec. 27, 2023

x86 SGDT emulation for Wine

by Elizabeth Figura

Hello all, There is a Windows 98 program, a game called Nuclear Strike, which wants to do some amount of direct VGA access. Part of this is port I/O, which naturally throws SIGILL that we can trivially catch and emulate in Wine. The other part is direct access to the video memory at 0xa0000, which in general isn't a problem to catch and virtualize as well. However, this program is a bit creative about how it accesses that memory; instead of just writing to 0xa0000 directly, it looks up a segment descriptor whose base is at 0xa0000 and then uses the %es override to write bytes. In pseudo-C, what it does is: int get_vga_selector() { sgdt(&gdt_size, &gdt_ptr); sldt(&ldt_segment); ++gdt_size; descriptor = gdt_ptr; while (descriptor->base != 0xa0000) { ++descriptor; gdt_size -= sizeof(*descriptor); if (!gdt_size) break; } if (gdt_size) return (descriptor - gdt_ptr) << 3; descriptor = gdt_ptr[ldt_segment >> 3]->base; ldt_size = gdt_ptr[ldt_segment >> 3]->limit + 1; while (descriptor->base != 0xa0000) { ++descriptor; ldt_size -= sizeof(*descriptor); if (!ldt_size) break; } if (ldt_size) return (descriptor - ldt_ptr) << 3; return 0; } Currently we emulate IDT access. On a read fault, we execute sidt ourselves, check if the read address falls within the IDT, and return some dummy data from the exception handler if it does [1]. We can easily enough implement GDT access as well this way, and there is even an out-of-tree patch written some years ago that does this, and helps the game run. However, there are two problems that I have observed or anticipated: (1) On systems with UMIP, the kernel emulates sgdt instructions and returns a consistent address which we can guarantee is invalid. However, it also returns a size of zero. The program doesn't expect this (cf. the way the loop is written above) and I believe will effectively loop forever in that case, or until it finds the VGA selector or hits invalid memory. I see two obvious ways to fix this: either adjust the size of the fake kernel GDT, or provide a switch to stop emulating and let Wine handle it. The latter may very well a more sustainable option in the long term (although I'll admit I can't immediately come up with a reason why, other than "we might need to raise the size yet again".) Does anyone have opinions on this particular topic? I can look into writing a patch but I'm not sure what the best approach is. (2) On 64-bit systems without UMIP, sgdt returns a truncated address when in 32-bit mode. This truncated address in practice might point anywhere in the address space, including to valid memory. In order to fix this, we would need the kernel to guarantee that the GDT base points to an address whose bottom 32 bits we can guarantee are inaccessible. This is relatively easy to achieve ourselves by simply mapping those pages as noaccess, but it also means that those pages can't overlap something we need; we already go to pains to make sure that certain parts of the address space are free. Broadly anything above the 2G boundary *should* be okay though. Is this feasible? We could also just decide we don't care about systems without UMIP, but that seems a bit unfortunate; it's not that old of a feature. But I also have no idea how hard it would be to make this kind of a guarantee on the kernel side. This is also, theoretically, a problem for the IDT, except that on the machines I've tested, the IDT is always at 0xfffffe0000000000. That's not great either (it's certainly caused some weirdness and confusion when debugging, when we unexpectedly catch an unrelated null pointer access) but it seems to work in practice. --Zeb [1] https://source.winehq.org/git/wine.git/blob/HEAD:/dlls/krnl386.exe16/ instr.c#l702

Dec. 27, 2023

Wine staging 9.0-rc3 release

by Alistair Leslie-Hughes

Binary packages for various distributions will be available from: https://www.winehq.org/download Summary since last release * Rebased to current wine 9.0-rc3 (498 patches are applied to wine vanilla) Upstreamed (Either directly from staging or fixed with a similar patch). * None Added: * None Updated: * None Where can you help * Run Steam/Battle.net/GOG/UPlay/Epic * Test your favorite game. * Test your favorite applications. * Improve staging patches and get them accepted upstream. * Suggest patches to be included in staging. As always, if you find a bug, please report it via https://bugs.winehq.org Best Regards Alistair.

Dec. 23, 2023

Re: vkd3d request

by Giovanni Mascellani

Hi, Il 07/12/23 21:13, Alistair Leslie-Hughes ha scritto: > Would it be possible to include a file of all games/applications that > have fixes when they are committed. It's too hard to track unless you > really doing serious work with it. It looks like a sensible request. I usually mention games which are fixed by a commit in the commit message, but it's true that having a list somewhere is easier than scanning all the commits in a new version. Gio.

Dec. 18, 2023

Wine staging 9.0-rc2 release

by Alistair Leslie-Hughes

Binary packages for various distributions will be available from: https://www.winehq.org/download Summary since last release * Rebased to current wine 9.0-rc2 (498 patches are applied to wine vanilla) Upstreamed (Either directly from staging or fixed with a similar patch). * None Added: * [45455] setupapi: Add stub for DriverStoreFindDriverPackageW. * [55282] where: Implement search with default options. Updated: * vkd3d-latest to latest. Where can you help * Run Steam/Battle.net/GOG/UPlay/Epic * Test your favorite game. * Test your favorite applications. * Improve staging patches and get them accepted upstream. * Suggest patches to be included in staging. As always, if you find a bug, please report it via https://bugs.winehq.org Best Regards Alistair.

Dec. 16, 2023

Re: Code freeze for Wine 9.0

by Alexandre Julliard

Folks, 9.0-rc1 has been released so we are now officially in code freeze. Please make sure to test your favorite apps and report issues and regressions. During code freeze, only small, targeted bug fixes will be accepted. When sending fixes, please make sure to mention the bug or the app that is fixed by the change. Note that changes to the tests are still freely accepted, so this is a good opportunity to fix Windows test failures. In particular, it would be really nice to make progress on the Gitlab CI Windows test runs that still have about a dozen systematic failures: https://test.winehq.org/data/93f7ef86701f0b5f0828c8e0c4581b00873a7676/win21… As usual, I'll be making rc releases every week until the final 9.0, which is expected some time around mid-January. Thanks to everyone for helping us make 9.0 as good as possible! -- Alexandre Julliard julliard(a)winehq.org

Dec. 10, 2023

Wine staging 9.0-rc1 release

by Alistair Leslie-Hughes

Binary packages for various distributions will be available from: https://www.winehq.org/download Summary since last release * Rebased to current wine 9.0-rc1 (493 patches are applied to wine vanilla) Upstreamed (Either directly from staging or fixed with a similar patch). * bcrypt: Allow multiple backends to coexist. * bcrypt: Implement BCryptSecretAgreement with libgcrypt. * bcrypt: Implement BCRYPT_KDF_HASH. Added: * None Updated: * vkd3d-latest to latest. * ntdll-DOS_Attributes * mfplat-streaming-support * ntdll-ForceBottomUpAlloc Where can you help * Run Steam/Battle.net/GOG/UPlay/Epic * Test your favorite game. * Test your favorite applications. * Improve staging patches and get them accepted upstream. * Suggest patches to be included in staging. As always, if you find a bug, please report it via https://bugs.winehq.org Best Regards Alistair.

Dec. 9, 2023

vkd3d request

by Alistair Leslie-Hughes

Hi All, Now that vkd3d 1.10 is out, I have a request for developers. Would it be possible to include a file of all games/applications that have fixes when they are committed. It's too hard to track unless you really doing serious work with it. The whole concept of including the latest vk3d3d in staging was to help with testing but if people don't know what to test, they wont. This would also, give a nice section in the ANNOUNCE as to what now works. :) Best Regards Alistair

Dec. 7, 2023

Re: Handling stack overflow

by Dāvis Mosāns

> [...] > > Since this is anti-cheat it is tricky, and things might not be what they seem. > > When you say "it messes something up", are you sure the hook is working > correctly to begin with? If not, that needs to be fixed rather than handling > the fallout exception. > I don't know if it's working correctly, but that's a separate issue, I'm looking into both things. > Do you know if the recursive invocation happens on Windows too? You mention > that the process is killed, so it doesn't sound like it is supposed to happen. > Yeah it's definitely not supposed to happen but Wine still could handle it better (hence this email). Here is small example that will deadlock Wine in exactly same way https://gist.github.com/davispuh/6c0ae7e5a10500a5b7a759b817e66214 In Windows such program will be killed and you'll get dialog with button to attach debugger. > Anti-cheat / Anti-debugger / DRM systems usually don't tell you nicely if they > think something is wrong. When they think you are trying to mess with them > they usually pretend to proceed for a while, do something else and then at a > much later point deliberately crash the process in a weird way. So your > exception handler recursion might be the Anti-cheat's underhanded attempt to > kill the process. True, but in this case it wasn't intended way, fixing LoadLibraryEx non-NULL hFile prevented stack overflow and deadlock https://gitlab.winehq.org/wine/wine/-/merge_requests/4587 ^ But I don't know if that was anti-debug/reversing trick or just working around some bigger issue. Game still doesn't work but now behaviour is way better because it doesn't deadlock and 100% of time presents game's internal crash dialog (it's used for all purposes, including if debugger is detected).

Dec. 4, 2023

Re: kernelbase: adding support for callback in CopyFileEx, and implementing CopyFile2

by Zeb Figura

On Friday, 1 December 2023 12:22:28 CST Nikolay Sivov wrote: > > There seem to have been multiple efforts to implement this functionality > > (in 2022 and 2013) but I can't tell find any reasoning for why they were > > not successful. Would implementing this be a desirable contribution? > > > > Sure, I recall we had a bug report for missing callbacks. That'd be bug 22690 and bug 22692.

Dec. 1, 2023