[Bug 50993] New: App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

Bug ID: 50993
           Summary: App compiled with address sanitizer don't work in Wine
           Product: Wine
           Version: 6.6
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: -unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

When trying to use in Wine app compiled with address sanitizer and MinGW LLVM,
then this app doesn't run and print
```
0104:err:module:LdrInitializeThunk "libclang_rt.asan_dynamic-x86_64.dll" failed
to initialize, aborting
0104:err:module:LdrInitializeThunk Initializing dlls for
L"Z:\mnt\Miecz\godot3.2\bin\a.exe" failed, status 80000003
```

To reproduce issue

Download and unpack Mingw-llvm(unlike Mingw GCC allows to build app with
address sanitizer)

https://github.com/mstorsjo/llvm-mingw/releases/download/20201020/llvm-mingw...

2. set variables(change PATH_TO_CHANGE)
export MINGW32_PREFIX="/PATH_TO_CHANGE/i686-w64-mingw32-"
export MINGW64_PREFIX="/PATH_TO_CHANGE/x86_64-w64-mingw32-"

3. Copy to current directory all files from folder which contains
`libclang_rt.asan_dynamic-x86_64.dll` inside unpacked mingw

4. Compile and try to open app

echo "int main(){if(2>1){return 1;}}" > a.cpp
/PATH_TO_CHANGE/x86_64-w64-mingw32-clang++ a.cpp -fsanitize="address"
wine a.exe

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

Roman Pišl  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |download, source

--- Comment #1 from Austin English  ---
It would be helpful if you could attach a source/binary that reproduces the
issue so that others don't need to set up a toolchain to test.

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #2 from Rafał Mikrut  ---
Created attachment 69834
  --> https://bugs.winehq.org/attachment.cgi?id=69834
Compiled binaries

Content of `program.zip` attachment:
- a.cpp - c++ source code
- a.exe - compiled app
- libclang_rt.asan_dynamic-x86_64.dll - asan library which probably causes
problems
- other files required by app

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #3 from Roman Pišl  ---
Created attachment 69964
  --> https://bugs.winehq.org/attachment.cgi?id=69964
Fix for clang address sanitizer

Hi, I tried this some time ago and the problem was that AddressSanitizer could
not intercept some functions (see
compiler-rt/lib/interception/interception_win.cpp).

Since LLVM/Clang can be now compiled under Wine (version 6.7) just by
installing LLVM12, CMake + MSVC libs, I gave it another chance
(wine-6.7-274-ge2aa30f21de, llvm trunk) and discovered that just a small fix
can make it work - see attached patch. With this fix, following code:

#include <malloc.h>
int main()
{
  char* buf = (char*) malloc(10);
  free(buf);
  return buf[1];
}

is properly detected while running in Wine as:
==428==ERROR: AddressSanitizer: heap-use-after-free on address 0x7eafe3e001b1
at pc 0x000140001056 bp 0x00000011fd60 sp 0x00000011fda8
READ of size 1 at 0x7eafe3e001b1 thread T0
    #0 0x140001055 in main Z:\mnt\data\src\llvm\llvm-test\main.cpp:8
...

So I think that this is not a Wine bug and it should be reported/fixed it in
upstream LLVM.

I also think that bug 49828 is a duplicate.

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #4 from Roman Pišl  ---
Created attachment 69979
  --> https://bugs.winehq.org/attachment.cgi?id=69979
Fix for clang dynamic address sanitizer

My previous post is valid for ASAN with static runtime. For dynamic runtime
(/MD), additional instructions have to be added to GetInstructionSize().

But only that is not sufficient and ASAN initialization still fails. That is
due to FindAvailableMemoryRange() is looking for a large free block of virtual
address space but all available address space is marked as MEM_RESERVE in Wine.

So a following quick hack..

diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_win.cpp
b/compiler-rt/lib/sanitizer_common/sanitizer_win.cpp
index 63c90785f270..b98cd78c55d4 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_win.cpp
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_win.cpp
@@ -372,8 +372,7 @@ uptr FindAvailableMemoryRange(uptr size, uptr alignment,
uptr left_padding,
     MEMORY_BASIC_INFORMATION info;
     if (!::VirtualQuery((void*)address, &info, sizeof(info)))
       return 0;
-
-    if (info.State == MEM_FREE) {
+    if (info.State == MEM_FREE || info.State == MEM_RESERVE) {
       uptr shadow_address = RoundUpTo((uptr)info.BaseAddress + left_padding,
                                       alignment);
       if (shadow_address + size < (uptr)info.BaseAddress + info.RegionSize)

pushes it little bit forward although it still fails.

And now:
ASAN_OPTIONS=asan.log WINEDEBUG=+relay wine main.exe 2>/dev/null 
makes ASAN work and asan.log contains correct output.

Seems to me making ASAN work under Wine could be feasible..

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #5 from Roman Pišl  ---
Created attachment 70041
  --> https://bugs.winehq.org/attachment.cgi?id=70041
Fix for Wine

I think now I understand it. Almost the whole virtual address space is covered
by a single view which also contains a 64bit reserved area
0x7ffffe000000-0x7fffffff0000 so it is marked as MEM_RESERVE in
get_free_mem_state_callback(). Attached quick fix solves the issue.

Interestingly, it also fixes bug 50735.

It seems that any 64bit application that is searching for a large chunk of a
free virtual space this way is possibly affected. So also these interceptors
and such..

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

François Gouget  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]
           Keywords|                            |patch

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

François Gouget  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |testcase

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

Roman Pišl  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #69964|0                           |1
        is obsolete|                            |
  Attachment #69979|0                           |1
        is obsolete|                            |
  Attachment #70041|0                           |1
        is obsolete|                            |

--- Comment #6 from Roman Pišl  ---
Created attachment 70233
  --> https://bugs.winehq.org/attachment.cgi?id=70233
Fix for clang address sanitizer

Since 0ec555e58ea9d5b33f4c825e96965ad0cb15d00f the problem with virtual address
space is fixed and the only remaining issue is function interception. I doubt
the Wine goal is to have exactly same preambles on all functions as on Windows
so this is probably a NOT OUR BUG and the fix should be made on clang side. I'm
attaching a fix that works for me. The only caveat on Wine side so far is
msvcrt/ntdll.strrchr. Clang needs 6 bytes for jump but there is a loop into
ntdll.strrchr+2. Either DECLSPEC_HOTPATCH or adding 'if (c == '\0') return
strchr(str, '\0');' to the beginning of the function fixes this. Hopefully this
can be useful for someone.

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

Jactry Zeng  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

Eric Pouech  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://bugs.winehq.org/sho
                   |                            |w_bug.cgi?id=52386

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

Bernhard Übelacker  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #7 from Bernhard Übelacker  ---
Created attachment 71677
  --> https://bugs.winehq.org/attachment.cgi?id=71677
Add some more instructions to GetInstructionSize.

I took Romans patch and Erics patch from 52386 and added a few more
instructions needed to run a x86_64-llvm-mingw generated test exe
with a regular x86_64-w64-mingw 10.2.1 "wine userland".
Also needed manually adding some link commands to winegcc.c and having
msvcrt/string.c:strlen compiled with "__attribute__((optimize("-O0")))".

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

Ralf Habacker  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #8 from Ralf Habacker  ---
Reported upstream at https://github.com/mstorsjo/llvm-mingw/issues/195

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #9 from Nikolay Sivov  ---
(In reply to Ralf Habacker from comment #8)
> Reported upstream at https://github.com/mstorsjo/llvm-mingw/issues/195

Could you try to get these changes in compiler-rt? llvm-mingw will then pull
from it, but it's not right to patch downstream project.

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

Martin Storsjö  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #10 from Martin Storsjö  ---
(In reply to Nikolay Sivov from comment #9)
> (In reply to Ralf Habacker from comment #8)
> > Reported upstream at https://github.com/mstorsjo/llvm-mingw/issues/195
>
> Could you try to get these changes in compiler-rt? llvm-mingw will then pull
> from it, but it's not right to patch downstream project.

+1, yes, post those patches for review at https://reviews.llvm.org (see
https://llvm.org/docs/Contributing.html, although I think patches like this
often go without an accompanying unit test). I wouldn't consider myself
authoritative to approve it there, but I can help find reviewers for it.

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

Nikolay Sivov  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         Depends on|                            |53288

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993
Bug 50993 depends on bug 53288, which changed state.

Bug 53288 Summary: Application compiled with MSVC 2022 ASan does not start, needs QueryVirtualMemoryInformation
https://bugs.winehq.org/show_bug.cgi?id=53288

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEEDINFO                    |RESOLVED
         Resolution|---                         |FIXED

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

Neko-san  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #11 from Bernhard Übelacker  ---
Tried to place a merge request with llvm-project here:
https://github.com/llvm/llvm-project/pull/113085

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #12 from Roman Pišl  ---
I tried the latest changes from PR 113085 with Wine 9.22 and it works with
windows_hook_rtl_allocators=false. If enabled it fails with:

AddressSanitizer: CHECK failed: asan_malloc_win.cpp:268 "((reallocFunc &&
heapSizeFunc && freeFunc && allocFunc)) != (0)" (0x0, 0x0) (tid=404)

I've sent a possible fix in
https://gitlab.winehq.org/wine/wine/-/merge_requests/6934

With this fix it also works with windows_hook_rtl_allocators enabled.

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #13 from Bernhard Übelacker  ---
(In reply to Roman Pišl from comment #12)
> I tried the latest changes from PR 113085 with Wine 9.22 and it works with
> windows_hook_rtl_allocators=false. If enabled it fails with:
>
> AddressSanitizer: CHECK failed: asan_malloc_win.cpp:268 "((reallocFunc &&
> heapSizeFunc && freeFunc && allocFunc)) != (0)" (0x0, 0x0) (tid=404)

Strangely I cannot remember getting this error in my testings.
Is there maybe additional console output before?

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #14 from Roman Pišl  ---
> Strangely I cannot remember getting this error in my testings.
> Is there maybe additional console output before?

It happens when an application compiled with asan tries to load a dll also
compiled with asan. The assertion fails on freeFunc == null. The interception
fails due to `jmp dword ptr` (0x25FF) instruction.

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #15 from Roman Pišl  ---
Created attachment 77518
  --> https://bugs.winehq.org/attachment.cgi?id=77518
Full log of windows_hook_rtl_allocators=true

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #16 from Bernhard Übelacker  ---
Created attachment 77521
  --> https://bugs.winehq.org/attachment.cgi?id=77521
dll-test_2024-12-15.txt

(In reply to Roman Pišl from comment #14)
> It happens when an application compiled with asan tries to load a dll also
> compiled with asan. The assertion fails on freeFunc == null. The
> interception fails due to `jmp dword ptr` (0x25FF) instruction.

Setting a breakpoint to HeapFree leads to several locations,
most with a similar jmp instruction.

And when running with ASAN_OPTIONS='verbosity=2' I get
this line between a few others:
   Failed to intercept ucrtbase.dll import HeapFree

So it seems not yet fully active in my environment.
Details in attached file.
Maybe a verbose log in your system shows a few less failing interceptions,
because of which I don't reach this situation?

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #17 from Roman Pišl  ---
> So it seems not yet fully active in my environment.
> Details in attached file.

I'll get back to this on the weekend, but if I read your log correctly:
- is from the environment where ASAN_OPTIONS=windows_hook_rtl_allocators=false,
otherwise `interception ucrtbase.dll import..` would not be reachable and it is
the same on my Linux box and I really don't know if that works on native
Windows
- the jmp in kernel32.dll is the issue that should be fixed in the MR
- my issue is triggered if HeapReAlloc is called and that probably does not
happen in your test case (and you must also set
windows_hook_rtl_allocators=true)

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

Bernhard Übelacker  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #77521|0                           |1
        is obsolete|                            |

--- Comment #18 from Bernhard Übelacker  ---
Created attachment 77524
  --> https://bugs.winehq.org/attachment.cgi?id=77524
dll-test_2024-12-03.txt

(In reply to Roman Pišl from comment #17)
> > So it seems not yet fully active in my environment.
> > Details in attached file.

This seems just because I had -O2, so the compiler optimized the unused
statements away. The new version attached triggers ASan much better.
> - the jmp in kernel32.dll is the issue that should be fixed in the MR
> - my issue is triggered if HeapReAlloc is called and that probably does not
>
> happen in your test case (and you must also set
> windows_hook_rtl_allocators=true)

Now I can reproduce the error message too.

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #19 from Roman Pišl  ---
> Now I can reproduce the error message too.

Good to hear. Please, test if the already accepted fix works for you.

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #20 from Bernhard Übelacker  ---
(In reply to Roman Pišl from comment #19)
> Good to hear. Please, test if the already accepted fix works for you.

Thanks for your fix, it works like a charm.

Without it ASan operates directly on something named HeapFree:
   interception_win: instruction at 0x6fffffa522c0: ff 25 9a e6 03 00 90 90
   0x00006fffffa522c0 <HeapFree+0>:     ff 25 9a e6 03 00

But with you fix ASan operates on something gdb identifies
as __wine_spec_imp_HeapFree:
   interception_win: instruction at 0x6fffffa4e5d0: 48 8d a4 24 00 00 00 00
   0x00006fffffa4e5d0 <__wine_spec_imp_HeapFree+0>:     48 8d a4 24 00 00 00 00

After all I wonder why ff 25 currently does not get used for interception.
The similar jmp 48 ff 25 gets used.
And previously there were two lines for ff 25,
just the latter got never reached:
  https://github.com/llvm/llvm-project/commit/a943922c0
One explanation could be it is used to detect already intercepted functions,
because test OverrideFunctionMultiplePatchingIsFailing would start to fail.
But I guess this starts to get offtopic here.

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #21 from Roman Pišl  ---
(In reply to Bernhard Übelacker from comment #20)
> But I guess this starts to get offtopic here.

Yes, let's keep it about the remaining changes in llvm, so it can be closed
once they are mainlined. But could you please test if interception of
ucrtbase.dll Heap* imports works on Windows and open new bug report eventually?

[Bug 50993] App compiled with address sanitizer don't work in Wine

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=50993

--- Comment #22 from Bernhard Übelacker  ---
(In reply to Roman Pišl from comment #21)
> ... But could you please test if interception of
> ucrtbase.dll Heap* imports works on Windows and open new bug report
> eventually?

I created #57498 with some more details.