Fixes a crash in PaintTool SAI when allocating more than 2GB of memory.
Signed-off-by: Elaine Lefler elaineclefler@gmail.com ---
From what I can tell, Sai basically requests an initial block of
memory from VirtualAlloc, then increments that pointer and calls VirtualAlloc again with a fixed address when it wants more ram. Blindly assuming that memory space is non-fragmented enough for the call to succeed. This is definitely incorrect program behavior, but it works on Windows, and after this tweak it's working on Wine too.
Heads up: I initially disabled the code path for all allocations, not just large ones. On macOS this works fine. On Linux it's broken. Somewhere, presumably in the X11 driver, we're truncating pointers to 32 bits. The change is safe for Windows apps, since Windows itself seems to avoid the lower 4GB region, even for functions like HeapAlloc.
I can't find any apparent cause for the truncation, other than converting HWNDs to XIDs, which doesn't seem to be the problem. I wonder if there might be a confusion between "long" and "long long" somewhere. Anyway, window data should never be large enough to trigger the new behavior, so the broken behavior stays under containment until it can be fixed properly. --- dlls/ntdll/unix/virtual.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/dlls/ntdll/unix/virtual.c b/dlls/ntdll/unix/virtual.c index 94b300c5057..eac9bcb1f83 100644 --- a/dlls/ntdll/unix/virtual.c +++ b/dlls/ntdll/unix/virtual.c @@ -1917,7 +1917,11 @@ static NTSTATUS map_view( struct file_view **view_ret, void *base, size_t size, alloc.top_down = top_down; alloc.limit = (void*)(get_zero_bits_mask( zero_bits ) & (UINT_PTR)user_space_limit);
- if (mmap_enum_reserved_areas( alloc_reserved_area_callback, &alloc, top_down )) + if ( +#ifdef _WIN64 + size < 2 * 1024 * 1024 && +#endif + mmap_enum_reserved_areas( alloc_reserved_area_callback, &alloc, top_down )) { ptr = alloc.result; TRACE( "got mem in reserved area %p-%p\n", ptr, (char *)ptr + size );