[Bug 57669] New: Suspicious behavior of rundll32.exe

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57669

Bug ID: 57669
           Summary: Suspicious behavior of rundll32.exe
           Product: Wine
           Version: 10.0-rc2
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: -unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

Created attachment 77844
  --> https://bugs.winehq.org/attachment.cgi?id=77844
trace - bad

I've upgraded a build environment with Wine/MSVC/LLCM from Debian Bullseye to
Bookworm and unfortunately it wasn't as easy as I expected. There is a problem
with rundll32.exe that I was finally able to isolate after lot of trial and
error.

The problem is on Debian Bookworm that the rundll32.exe process that is started
never reaches wWinMain (I put TRACE() at the beginning). I'm attaching the bad
trace log from Bookworm and the good trace from Bullseye.

A simple workaround for me now is to copy rundll32.exe from Bullseye.

Any ideas what to try next? Is this a hidden bug in Wine or Mingw compiler?

(setting component to unknown as rundll32 is missing in the list)

[Bug 57669] Suspicious behavior of rundll32.exe

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57669

--- Comment #1 from Roman Pišl  ---
Created attachment 77845
  --> https://bugs.winehq.org/attachment.cgi?id=77845
trace - good

[Bug 57669] Suspicious behavior of rundll32.exe

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57669

--- Comment #2 from Austin English  ---
Are the versions of wine/mingw/llvm different in these environments?

[Bug 57669] Suspicious behavior of rundll32.exe

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57669

--- Comment #3 from Roman Pišl  ---
(In reply to Austin English from comment #2)
> Are the versions of wine/mingw/llvm different in these environments?

The Wine prefix (with Windows version of toolchains to compile another project)
is always the same. All tested -rc versions have the same behavior, no matter
if I use precompiled devel binaries from Wine repository or compile Wine
locally, no difference (Bullseye ok, Bookworm bad).

[Bug 57669] Suspicious behavior of rundll32.exe

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57669

--- Comment #4 from Roman Pišl  ---
Next I will try to compile Wine with different versions of Mingw on Bookworm.

[Bug 57669] Suspicious behavior of rundll32.exe

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57669

--- Comment #5 from Alexandre Julliard  ---
What app are you running?

It looks like a pointer truncation bug in the parent process:

028c:0290:trace:virtual:NtAllocateVirtualMemory 0x60 (nil) 0000020e 1000
00000004
02b4:02b8:trace:virtual:NtAllocateVirtualMemory 0xffffffffffffffff (nil)
0000020e 1000 00000004
02b4:02b8:trace:virtual:map_view got mem in reserved area
0x7ffffe320000-0x7ffffe321000
02b4:02b8:trace:virtual:dump_view View: 0x7ffffe320000 - 0x7ffffe320fff
(valloc)
02b4:02b8:trace:virtual:dump_view       0x7ffffe320000 - 0x7ffffe320fff c-rw-
028c:0290:trace:virtual:NtQueryVirtualMemory (0x60, 0xfe320000, info_class=0,
0x51e6f4, 28, 0x51e6f0)

The 0x7ffffe320000 pointer is truncated. Is the parent a 32-bit process?

[Bug 57669] Suspicious behavior of rundll32.exe

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57669

--- Comment #6 from Roman Pišl  ---
(In reply to Alexandre Julliard from comment #5)
> The 0x7ffffe320000 pointer is truncated. Is the parent a 32-bit process?

Good catch, yes, the parent process is a 32bit MSBuild.exe. Maybe the previous
call to NtAllocateVirtualMemory() returns a 64bit address to a 32bit process?

[Bug 57669] Suspicious behavior of rundll32.exe

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57669

--- Comment #7 from Alexandre Julliard  ---
Does it work in new wow64 mode?

[Bug 57669] Suspicious behavior of rundll32.exe

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57669

--- Comment #8 from Roman Pišl  ---
(In reply to Alexandre Julliard from comment #7)
> Does it work in new wow64 mode?

Yes, new wow64 works!

[Bug 57669] Suspicious behavior of rundll32.exe

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=57669

--- Comment #9 from Roman Pišl  ---
Created attachment 77853
  --> https://bugs.winehq.org/attachment.cgi?id=77853
trace - new wow64