- Wine-devel - list.winehq.org

Re: Helping Wine use 64 bit Mesa OGL drivers for 32-bit Windows applications
by Derek Lesho 19 Oct '24

19 Oct '24

On 10/19/24 00:47, Faith Ekstrand wrote: > The timing here isn't great, unfortunately. I'd love to contribute > more to the discussion but I'm going on leave starting next week until > mid-Febuary so I won't be able to participate much until then. I'll > try to leave a few thoughts, though. Thanks for the comments! I'll actually also be quite busy until then (Uni), but maybe by February they'll be a consensus on the path forward 😀. > > On Fri, Oct 18, 2024 at 5:10 PM Derek Lesho <dlesho(a)codeweavers.com> > wrote: > > Hey everyone 👋, > > I'm Derek from the Wine project, and wanted to start a discussion > with > y'all about potentially extending the Mesa OGL drivers to help us > with a > functionality gap we're facing. > > Problem Space: > > In the last few years Wine's support for running 32-bit windows > apps in > a 64-bit host environment (wow64) has almost reached feature > completion, > but there remains a pain point with OpenGL applications: Namely that > Wine can't return a 64-bit GL implementation's buffer mappings to > a 32 > bit application when the address is outside of the 32-bit range. > > Currently, we have a workaround that will copy any changes to the > mapping back to the host upon glBufferUnmap, but this of course is > slow > when the implementation directly returns mapped memory, and > doesn't work > for GL_PERSISTENT_BIT, where directly mapped memory is required. > > A few years ago we also faced this problem with Vulkan's, which was > solved through the VK_EXT_map_memory_placed extension Faith drafted, > allowing us to use our Wine-internal allocator to provide the > pages the > driver maps to. I'm now wondering if an GL equivalent would also > be seen > as feasible amongst the devs here. > > Proposed solution: > > As the GL backend handles host mapping in its own code, only giving > suballocations from its mappings back to the App, the problem is a > little bit less straight forward in comparison to our Vulkan > solution: > If we just allowed the application to set its own placed mapping when > calling glMapBuffer, the driver might then have to handle moving > buffers > out of already mapped ranges, and would lose control over its own > memory > management schemes. > > Therefore, I propose a GL extension that allows the GL client to > provide > a mapping and unmapping callback to the implementation, to be used > whenever the driver needs to perform such operations. This way the > driver remains in full control of its memory management affairs, > and the > amount of work for an implementation as well as potential for bugs is > kept minimal. I've written a draft implementation in Zink using > map_memory_placed [1] and a corresponding Wine MR utilizing it > [2], and > would be curious to hear your thoughts. I don't have experience in > the > Mesa codebase, so I apologize if the branch is a tad messy. > > It's an interesting approach, to be sure. I don't mean that as a bad > or good thing as I haven't given this enough thought with GL in mind > to have a better, more well thought out plan. > > The most obvious issue that jumps out to me is that we really want > that callback to be set before anyone ever maps a buffer that might > possibly get exposed to the client and we want it to never change. If > this were Vulkan, we'd have you provide it at vkCreateDevice() time. > But this is GL where everybody loves a big mutable state object. If we > do go with callbacks (and it's still not 100% clear to me what the > right choice is), we'd want them to be somehow set-once and set before > any buffers are created. I'm not 100% sure how you'd spec that or how > we'd enforce it. There may be some precedent for this somewhere in GL > (no_error, maybe?) but I'm not sure. Right, in the case of Zink I was just lucky it doesn't happen to map anything upon context creation. If I understand what you mean by using a no_error like approach correctly, I think that should definitely work, I think we would then just somewhat-awkwardly want to pass through the callback address as two context attributes, one for the lower and one for the higher part of the address. However, even if we find a way to make the mapping callback's global on the context level, that then still requires from the driver that they relegate mappings in these contexts to dedicated memory pools. This might be desired, in order to allow other GL clients in the process to continue using the full 64-bit address space, although in practice I don't think Wine uses any libraries that create GL contexts (*other than potentially GStreamer which we are moving away from). If we just want to keep it simple and workaround the GL context, maybe Wine could just export its allocator in ntdll.so in a way that where Mesa could then directly call it when present. That way we could even avoid the need for a GL extension that will at the end of the day probably only be used by Wine, plus avoid the awkwardness of not wanting to map anything in the driver until we get mapping callback. > > The other question that comes to mind is when exactly we'd be expected > to use these things. Obviously, we need to do so for any map that may > be exposed to the client. However, it's not always easy to do that > because you don't know at buffer create time whether or not it will be > persistently mapped. A driver is likely to have all sorts of internal > mappings for things and, while those can come from one of those > ranges, it'll burn more of that precious 32-bit address space than > needed. This gets worse when you take sub-allocation into account. If > we're okay with all buffer mappings going down the client-request path > then it's probably okay. The driver just might need an extra bit in > its buffer cache key. Yeah, it would definitely be a nice bonus to the new path if drivers kept their internal mappings outside of 32-bit address space, but as far as I can see I don't think this affects our choice of interface, as mesa should be able to handle client visible buffers differently from driver-internal ones without the help of the client. > > I'm also sitting here trying to come up with some plan that would let > us do this more like Vulkan and I'm having trouble coming up with one > that works. GL has no concept of "create time". We could theoretically > do something where we flush everything, copy the data to a new > placed-mappable buffer and then continue on but that's gonna suck. Yeah, and then you also still have to define how you create this new placed-mappable buffer, and there isn't really a small-neat solution for this. You could define a GL_OUT_OF_PLACED_MEMORY error in glMapBuffer, and then have another entry point by which the app feeds the driver pages, but then you run into the problem of unmapping, which as far as I can see is often performed asynchronously. > > I think that's all that comes to mind immediately. As I said at the > top, I'm happy to talk more in a few months. Best of luck until then! > > ~Faith > > In theory, the only requirement from drivers from the extension > would be > that glMapBuffer always return a pointer from within a page allocated > through the provided callbacks, so that it can be guaranteed to be > positioned within the required address space. Wine would then use > it's > existing workaround for other types of buffers, but as Mesa seems to > often return directly mapped buffers in other cases as well, Wine > could > also avoid the slowdown that comes with copying in these cases as > well. > > Why not use Zink?: > > There's also a proposal to use a 32-bit PE build of Zink in Wine > bypassing the need for an extension; I brought this to discussion in > this Wine-Devel thread last week [3], which has some arguments > against > this approach. > > > For cases where Zink is being used on the host (this is the current > plan for Nouveau going forward), doing Zink in Windows may not be a > bad idea. However, I agree that it may not be the best idea to rely on > that plan. > > If any of you have thoughts, concerns, or questions about this > potential > approach, please let me know, thanks! > > 1: > https://gitlab.freedesktop.org/Guy1524/mesa/-/commits/placed_allocation > > 2: https://gitlab.winehq.org/wine/wine/-/merge_requests/6663 > > 3: https://marc.info/?t=172883260300002&r=1&w=2 > <https://marc.info/?t=172883260300002&r=1&w=2> >

1 0

Helping Wine use 64 bit Mesa OGL drivers for 32-bit Windows applications
by Derek Lesho 18 Oct '24

18 Oct '24

Hey everyone 👋, I'm Derek from the Wine project, and wanted to start a discussion with y'all about potentially extending the Mesa OGL drivers to help us with a functionality gap we're facing. Problem Space: In the last few years Wine's support for running 32-bit windows apps in a 64-bit host environment (wow64) has almost reached feature completion, but there remains a pain point with OpenGL applications: Namely that Wine can't return a 64-bit GL implementation's buffer mappings to a 32 bit application when the address is outside of the 32-bit range. Currently, we have a workaround that will copy any changes to the mapping back to the host upon glBufferUnmap, but this of course is slow when the implementation directly returns mapped memory, and doesn't work for GL_PERSISTENT_BIT, where directly mapped memory is required. A few years ago we also faced this problem with Vulkan's, which was solved through the VK_EXT_map_memory_placed extension Faith drafted, allowing us to use our Wine-internal allocator to provide the pages the driver maps to. I'm now wondering if an GL equivalent would also be seen as feasible amongst the devs here. Proposed solution: As the GL backend handles host mapping in its own code, only giving suballocations from its mappings back to the App, the problem is a little bit less straight forward in comparison to our Vulkan solution: If we just allowed the application to set its own placed mapping when calling glMapBuffer, the driver might then have to handle moving buffers out of already mapped ranges, and would lose control over its own memory management schemes. Therefore, I propose a GL extension that allows the GL client to provide a mapping and unmapping callback to the implementation, to be used whenever the driver needs to perform such operations. This way the driver remains in full control of its memory management affairs, and the amount of work for an implementation as well as potential for bugs is kept minimal. I've written a draft implementation in Zink using map_memory_placed [1] and a corresponding Wine MR utilizing it [2], and would be curious to hear your thoughts. I don't have experience in the Mesa codebase, so I apologize if the branch is a tad messy. In theory, the only requirement from drivers from the extension would be that glMapBuffer always return a pointer from within a page allocated through the provided callbacks, so that it can be guaranteed to be positioned within the required address space. Wine would then use it's existing workaround for other types of buffers, but as Mesa seems to often return directly mapped buffers in other cases as well, Wine could also avoid the slowdown that comes with copying in these cases as well. Why not use Zink?: There's also a proposal to use a 32-bit PE build of Zink in Wine bypassing the need for an extension; I brought this to discussion in this Wine-Devel thread last week [3], which has some arguments against this approach. If any of you have thoughts, concerns, or questions about this potential approach, please let me know, thanks! 1: https://gitlab.freedesktop.org/Guy1524/mesa/-/commits/placed_allocation 2: https://gitlab.winehq.org/wine/wine/-/merge_requests/6663 3: https://marc.info/?t=172883260300002&r=1&w=2

1 0

Wine staging 9.20 release
by Alistair Leslie-Hughes 18 Oct '24

18 Oct '24

Binary packages for various distributions will be available from: https://www.winehq.org/download Summary since last release * Rebased to current wine 9.20 (357 patches are applied to wine vanilla) Upstreamed (Either directly from staging or fixed with a similar patch). * None. Removed (No longer required). * None Added: * [33190]: win32u - Fix rendering of the rotated and scaled text. Updated: * vkd3d-latest Where can you help * Run Steam/Battle.net/GOG/UPlay/Epic * Test your favorite game. * Test your favorite applications. * Improve staging patches and get them accepted upstream. * Suggest patches to be included in staging. As always, if you find a bug, please report it via https://bugs.winehq.org Best Regards Alistair.

1 0

Trouble compiling Winelib DLL for 64-bit on OpenSUSE
by aleithner＠level101.at 18 Oct '24

18 Oct '24

Hi, I have previously asked about the problem I'm describing below on openSUSE's forums [1], StackOverflow [2] and Wine's forums [3], where dimesio told me that I could try to ask here. If this is the wrong place to ask, I apologise. Over the past few weeks, I've been re-implementing a DLL using Winelib to be able to leverage Unix sockets to communicate with a Linux process outside of Wine. For context, I'm on a 64-bit machine running openSUSE Tumbleweed. I've had success compiling the DLL as a 32-bit object using the command line winegcc -m32 -L/usr/lib64/wine/i386-unix -shared -o myproject.dll myproject_main.c myproject.spec This DLL (myproject.dll.so) works as expected, i.e. 32-bit Windows programs running under Wine can load and execute the DLL just fine. (I've found specifying -L/usr/lib64/wine/i386-unix to be necessary since WineGCC didn't pick up Wine's library path by itself.) Now, I've been trying to compile this same project as a 64-bit object using winegcc -L/usr/lib64/wine/x86_64-unix -shared -o myproject64.dll myproject_main.c myproject.spec and I'm facing the following WineGCC output: /usr/lib64/gcc/x86_64-suse-linux/14/../../../../x86_64-suse-linux/bin/ld: cannot find -ladvapi32: No such file or directory /usr/lib64/gcc/x86_64-suse-linux/14/../../../../x86_64-suse-linux/bin/ld: cannot find -luser32: No such file or directory /usr/lib64/gcc/x86_64-suse-linux/14/../../../../x86_64-suse-linux/bin/ld: cannot find -lwinecrt0: No such file or directory /usr/lib64/gcc/x86_64-suse-linux/14/../../../../x86_64-suse-linux/bin/ld: cannot find -lkernel32: No such file or directory /usr/lib64/gcc/x86_64-suse-linux/14/../../../../x86_64-suse-linux/bin/ld: cannot find -lntdll: No such file or directory collect2: error: ld returned 1 exit status winegcc: /usr/bin/gcc failed As I've discovered, openSUSE's wine-devel package does not provide any archive files in /usr/lib64/wine/x86_64-unix, whereas wine-devel-32bit does provide archive files under /usr/lib/wine/i386-unix (symlinked with the same directory in /usr/lib64). Interestingly, I've inspected Wine's Fedora packages [4] and haven't found these archive files for 32-bit but only for 64-bit; so, the Fedora packages seem to have the opposite "problem." Now, I've tried to use the archive files provided by openSUSE's 64-bit wine-devel package under /usr/lib64/wine/x86_64-windows, but of course they're PE binaries so LD complains it can't link to my Winelib DLL: /usr/bin/ld: relocatable linking with relocations from format pe-x86-64 (/usr/lib64/wine/x86_64-windows/libwinecrt0.a(debug.o)) to format elf64-x86-64 (tmp66fd686a/myproject64-00000001.spec-00000001.o) is not supported winebuild: /usr/bin/ld failed with status 1 winegcc: /usr/bin/winebuild failed I feel like I should be able to compile this Winelib DLL as both a 32-bit and a 64-bit object respectively, on any supported system. (I haven't found any info on this skimming through the Winelib User's Guide [5].) Is this assumption wrong? If so, what's the correct way of building Winelib DLLs? If not, how can I get a 64-bit version of this DLL? Any advice on what I might be misunderstanding or doing wrong or what might be the cause of this problem would be greatly appreciated. Best regards, Alexander Leithner [1]: https://forums.opensuse.org/t/wines-64-bit-archive-files-are-missing/179038 [2]: https://stackoverflow.com/questions/79058718/whats-the-correct-way-of-build… [3]: https://forum.winehq.org/viewtopic.php?t=39513 [4]: e.g. https://dl.winehq.org/wine-builds/fedora/40/x86_64/wine-staging-9.19-1.1.x8… under /opt/wine-staging/lib64/wine inside the package [5]: https://gitlab.winehq.org/wine/wine/-/wikis/Winelib-User's-Guide

6 5

Discussing paths forward for OpenGL mappings on Wow64
by Derek Lesho 15 Oct '24

15 Oct '24

Hi everyone, I'm creating this thread to start a discussion about how we want to complete GL support on Wow64, in the hopes that, if we want a MESA extension for this, we can get it to them sooner rather than later. The core of the problem is same that we faced for Vulkan: namely that host visible mappings from graphics APIs may be located above 32-bit memory, in which case we can't pass them to the application as is. In the current implementation that Rémi wrote a few years ago, we use a temporary buffer for glMapBuffer, and manually have to copy the data from it back to the real mapping on glUnmapBuffer. This solution works for most applications, but is slower than direct writes, and doesn't allow use of the GL_MAP_PERSISTENT_BIT glMapBuffer flag, where the application doesn't have to unmap the memory before usage by the API. Because of this problem, we can't support GL versions >= 4.4 in this path, which admittedly isn't a huge problem considering wined3d now supports Vulkan, but does rule out some native GL apps/games from running on the new path. I'll go over a few potential solutions: - "Just Use Zink": this idea has been floated for a while, and would be to use a PE build of Zink, the MESA gallium driver on top of Vulkan, which would then automatically make use of our VK_EXT_map_memory_placed integration in winevulkan and bypass the problem. Rémi has a branch with a draft solution for this [1] The advantage of approach is that it doesn't require any more extensions to any more APIs, but the disadvantage is that Wine would have to worry about keeping a separate version of Mesa up to date and support for building the required c++ components of mesa to its build system, as can be seen in the commits. - GL extension with placed memory allocation callback: In this case, Wine provides a map and unmap callback for the GL implementation to use in creating the pages it needs for GPU-mappings. In comparison to the PE Zink solution, we can continue to use system libraries, and maintain fast buffer IO, as long as the glMapBuffer implementation returns the mapped ptr directly. The main downside for this solution is of course the introduction of a new extension to the mostly dormant GL API, but here it would be possible to just use a VK_EXT_map_memory leveraging Zink like in the first solution, only this time on the Unix side. For this solution I've created a draft: Wine MR [2] Mesa Branch with Zink implementation [3] - glMapBuffer extension which we send our placed allocation: The idea here is that slightly extended glMapBuffer could be sent a flag to use Wine mapping, avoiding callbacks and targeting the problem where it manifests for Wine (when we get an address out of 32-bit space). This was briefly discussed on the LGD discord server, but it won't work well, due to the fact that a GL buffer is usually only a suballocation of a memory mapping, and often has already been assigned to another pool of memory by the time glMapMemory is called. Mesa would have to add a considerable sized implementation setting up custom-allocator pools, and be able to move buffers between them to implement this. - glBufferStorage extension to ease Mesa implementation: The next logical conclusion based on the problems of the last solution is that the Mesa implementation should get the information about any custom allocation at buffer creation time. The closest equivalent to this is the glBufferStorage entry, where we could create a new type of memory that we would ask the implementation to put the buffer in. For this solution we'd either have to implement custom memory allocator or introduce more API entries in order to allow the driver to convey to the application what size mappings it prefers. This would be very unwieldy, and wouldn't solve speed up the slow buffer copies for legacy buffers which don't use ARB_buffer_storage. - finally, going off the last possibility, if we're willing to create our own buffer allocator (we probably shouldn't, as the driver probably knows best), we could use the already existing EXT_external_objects extensions to import Vulkan memory objects, from which our allocator would allocate slices for the buffers. In glMapBuffer we would then return the Vulkan placed memory mapping. If I got anything wrong or overlooked anything, please let me know. If we decide on a solution that involves mesa, I would gladly then include the mesa-dev list to see if they have opinions for/against any of the proposals. Thanks, Derek 1: https://gitlab.winehq.org/rbernon/wine/-/tree/wip/just-use-zink 2: https://gitlab.winehq.org/wine/wine/-/merge_requests/6663 3: https://gitlab.freedesktop.org/Guy1524/mesa/-/commits/placed_allocation

5 7

wined3d surface handle buffer overflow
by Fabian Maurer 11 Oct '24

11 Oct '24

Hello, I'm currently investigating a wined3d regression that leads to an assert - bug 57300. In short, there is memorys corruption in "d3d_device2_SwapTextureHandles" when copying a "global handle table" handle into the "device handle table". The "device handle table" is only 64 entries long, but the handle is bigger than that - writing out of bounds. As I see, it all surfaces get allocated into the global handle table. Then in d3d_device2_SwapTextureHandles they get copied into the device handle table. Once in "CreateStateBlock" and once in "EndStateBlock". The device handle table only contains the wined3d state block handle. If those assumptions are correct, I have a few questions: 1) Isn't there a risk of overriding the state block handle when swapping the texture with handle 1? 2) Why are the handles only copied, completely bypassing the ddraw_allocate_handle function? Doesn't this mess up the next call to "ddraw_allocate_handle"? A potential solution to the buffer overflow would be the increase the size of the table if the handle doesn't fit. But I don't know if that's the correct course of action, and I don't know what to do with the table size / count values. Or maybe there' something else wrong here that I'm not seeing, insights would be appreciated. Regards, Fabian Maurer

2 1

Upcoming vkd3d 1.14 release
by Henri Verbeet 10 Oct '24

10 Oct '24

Hi all, As previously announced, the planned release date for vkd3d 1.14 is November 21. That means the public API will become frozen two weeks from now on October 24, and features will be frozen after November 7. As usual, if you're an upstream vkd3d developer, please do the following: - Keep an eye on Bugzilla for any reported regressions. - Review any API changes since the previous release. - Verify all vkd3d tests pass on your hardware/setup, and please send fixes if they don't. - Review and improve the documentation. - Apply your best judgment about which patches are appropriate for the current stage of the release process. If you're a downstream user of vkd3d, please test your applications for any regressions, and file bug reports if you find any. The planned release date for vkd3d 1.15 is February 19. Regards, Henri

1 0

Wine staging 9.19 release
by Alistair Leslie-Hughes 05 Oct '24

05 Oct '24

Binary packages for various distributions will be available from: https://www.winehq.org/download Summary since last release * Rebased to current wine 9.19 (354 patches are applied to wine vanilla) Upstreamed (Either directly from staging or fixed with a similar patch). * d3dx9_36: Add format description for X8L8V8U8 for format conversions. Removed (No longer required). * None Added: * None Updated: * vkd3d-latest Where can you help * Run Steam/Battle.net/GOG/UPlay/Epic * Test your favorite game. * Test your favorite applications. * Improve staging patches and get them accepted upstream. * Suggest patches to be included in staging. As always, if you find a bug, please report it via https://bugs.winehq.org Best Regards Alistair.

1 0

ntdll WRITECOPY dropped from staging
by Fabian Maurer 28 Sep '24

28 Sep '24

Hello, I noticed that the ntdll WRITECOPY changes got mostly dropped from staging in commit https://github.com/wine-staging/wine-staging/commit/ 16dce521242b3f4a52cd2c8799ce6fe464fac83c > The current state even after multiple iterations, the whole > WRITECOPY implemenet needs differently. Upstream will need to > do this. As I understand it, the implementation marks WRITECOPY pages as read-only on the unix side, handling the access violation and marking it as read-write afterwards. (Correct me if I'm wrong) What is wrong with the underlying concept, how should it be implemented properly? Regards, Fabian Maurer

2 1

Carry flag state after a system function call
by robert.ayrapetyan＠gmail.com 25 Sep '24

25 Sep '24

Hello, some legacy applications depend on the carry flag state after calling system functions. I've noticed that certain apps work correctly when using Wine installed from packages, but fail when Wine is built from sources. After investigating, I found that Wine from Debian packages is compiled with certain flags that stabilize ESP at the end of system function calls (e.g., sub ESP, 0x10), which, as a side effect, clears the carry flag, making the application happy. I tried using fno-omit-frame-pointer, but it generates lea instructions, which don't impact the carry flag. Has anyone else encountered same behavior? How frequently do older applications rely on the carry flag after system function calls? Also, which specific flags are used to compile the Wine version available in Debian packages?

5 5