[Bug 56876] New: Paint Tool SAIv2 VirtualAlloc invalid address on commit

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=56876

Bug ID: 56876
           Summary: Paint Tool SAIv2 VirtualAlloc invalid address on
                    commit
           Product: Wine
           Version: 9.11
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: ntdll
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

When using Paint Tool SAIv2 (any recent version, Preview.2024.05.22 for
example), the program will often start emitting errors when a few files are
opened and random scatter brushes are selected (brushes that open bitmap files
to draw themselves). After this point, the program can quickly become unstable
and inoperable, further opening of files may fail, and attempting to save files
in this state may cause file corruption. I've never experienced this issue when
using SAI in Windows 10, so I think it is more likely an issue within Wine than
SAI itself.

What I think is causing the errors:
Although the errors seems to be related to files being opened, it seems to be
caused by subsequent memory allocations. Checking SAI's error log, it says
"!WinError(487): Invalid address." When running with WINDDEBUG=+all, I found
that an invalid address error (0x000001E7, 487) is reported after the program
attempts to commit a page of memory with VirtualAlloc:

For this particular example, the program likely attempted to write to a
reserved address 177FFF8, so then a SEH for EXCEPTION_ACCESS_VIOLATION occured.
The SEH then attempted to commit 177FFF8 with VirtualAlloc, but it failed, and
then GetLastError returned 000001e7 ERROR_INVALID_ADDRESS:
38822.915:0020:01b8:Call
KERNEL32.VirtualAlloc(0177fff8,00001000,00001000,00000004) ret=14019f0d2
38822.915:0020:01b8:Ret  KERNEL32.VirtualAlloc() retval=00000000 ret=14019f0d2

This should have been a valid address to commit, which should have returned the
next lowest page, for example 177f000.

The internal call to ntdll.NtAllocateVirtualMemory returned c0000019, which is
STATUS_NOT_MAPPED_VIEW.

Looking earlier into the log, I can confirm that the region should have indeed
been reserved:
38744.422:0020:0024:Call
KERNEL32.VirtualAlloc(00000000,00400000,00002000,00000004) ret=1401a0a0e
38744.422:0020:0024:Ret  KERNEL32.VirtualAlloc() retval=01380000 ret=1401a0a0e
This region 01380000 - 1780000 contains address 177FFF8 (8 bytes from the end
of the last page).

I could not find any indication that this reserved region is ever freed by
VirtualFree. Therefore, I think that this may indicate that the views_tree used
by find_view had become damaged somehow:
https://github.com/wine-mirror/wine/blob/master/dlls/ntdll/unix/virtual.c#L4...

I've compressed the massive log file, but it's still too large to attach, so
I've uploaded it here:
https://mega.nz/file/ET5S1LzY#WyVvta3n4QxeLjmzkyJqWmU5ZdgODWh4jILvNg32_rE

[Bug 56876] Paint Tool SAIv2 VirtualAlloc invalid address on commit

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=56876

--- Comment #1 from cromachina  ---
I've examined the issue several different times with logging, and quite often
does the error seem to occur when the reserved address to be committed by
VirtualAlloc is seemingly the in the very last page of the reserved region.
Aside from views_tree being potentially corrupted, it makes me think there
could be a bug when computing page/region boundary in find_view, somewhere
around here:
https://github.com/wine-mirror/wine/blob/master/dlls/ntdll/unix/virtual.c#L1...

[Bug 56876] Paint Tool SAIv2 VirtualAlloc invalid address on commit

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=56876

Fabian Maurer  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]