https://bugs.winehq.org/show_bug.cgi?id=44375
--- Comment #18 from Kai Krakow kai@kaishome.de --- (In reply to William Pierce from comment #17)
My pmap parsing from https://github.com/doitsujin/dxvk/issues/1318#issuecomment-574464662 give more credit to Kai Krakow's comment 8 with respect to the driver mapping address space into the process.
As far as I remember, this problem builds around the driver using glibc functions to allocate memory into the address space. But glibc uses a different strategy than Windows (I think one works bottom-up, the other top-down). Wine emulates the Windows strategy by implementing its own allocation manager. So both strategies work against each other. This cannot really be worked around as the callbacks into the driver don't really know that they are called from a wine context, or more clearly: glibc doesn't know and doesn't care (which is fine, it's a Linux lib). This also affects other libraries which allocate memory outside of wine scope. And it's mostly a problem only in 32-bit context because address space is exhausted much earlier. The work-around by NVIDIA was passing memory through SHM to swap out 32-bit address space into 64-bit system memory. The link is here:
https://www.nvidia.com/Download/driverResults.aspx/92712/en-us
Added a new system memory allocation mechanism for large allocations in the OpenGL driver. This mechanism allows unmapping the allocation from the process when it is not in use, making more virtual address space available to the application. It is enabled by default on 32 bit OpenGL applications with Linux 3.11+ and glibc 2.19+. Memory allocated this way will consume space in /dev/shm. Setting the environment variable __GL_DevShmPageableAllocations to 2 will disable this feature
From this perspective my best guess is: Vulkan applications should have the
best chance of working better here because memory management will be done mostly by the application itself (or DXVK in that case). But still, that's an issue because the driver cannot work on DXVK allocations only, it still needs to do its own. DXVK uses a lot of driver calls where this is true.
Also, I'm not sure if "__GL_DevShmPageableAllocations" also applies to Vulkan or only to OpenGL. In the latter case, it would explain why WineD3D works better than DXVK. One could try to set it to value 2 and retest with WineD3D to see if it fails earlier now. Tho, one could probably not reverse that argument and conclude that Vulkan doesn't use that feature then. NVIDIA devs should know.