[Bug 38886] New: AArch64 platforms: ABI Problems wrt varargs (needs arm64 specific __builtin_ms_va_list)
https://bugs.winehq.org/show_bug.cgi?id=38886
Bug ID: 38886 Summary: AArch64 platforms: ABI Problems wrt varargs (needs arm64 specific __builtin_ms_va_list) Product: Wine Version: 1.7.46 Hardware: aarch64 URL: http://go.microsoft.com/fwlink/p/?LinkId=536682 OS: Linux Status: NEW Keywords: download Severity: normal Priority: P2 Component: -unknown Assignee: wine-bugs@winehq.org Reporter: nerv@dawncrow.de CC: focht@gmx.net, nerv@dawncrow.de Distribution: ---
Hi,
running e.g. makecat.exe or midl.exe gives backtraces which suggest varargs problems.
relevant part: (stripped) Backtrace: =>0 0x0000007f84d4aa04 raise_segv_exception(rec=0x7f83e8fa18, context=0x7f83e8f908) in ntdll (0x0000007f83e8fab0) 1 0x0000007f83d65864 pf_printf_w+0x8cb(pf_puts=0x7f83d61b18, puts_ctx=0x7f83e8fc70, fmt=<is not available>, locale=<is not available>, positional_params=0, invoke_invalid_param_handler=0x740020, pf_args=0x78003000200068, args_ctx=0x2d0020003a0000, valist=0x7f83e8fcb0) in msvcrt (0x0000007f83e8fab0) 2 0x0000007f83d65864 pf_printf_w+0x8cb(pf_puts=0x7f83d67ec0, puts_ctx=0x7f83e8fc70, fmt=<is not available>, locale=<is not available>, positional_params=0, invoke_invalid_param_handler=0, pf_args=0x7f83d617d8, args_ctx=0x0(nil), valist=0x7f83e8fcc8) in msvcrt (0x0000007f83e8fac0) 3 0x0000007f83d65864 pf_printf_w+0x8cb(pf_puts=0x7f836822c0, puts_ctx=0x0(nil), fmt=<is not available>, locale=<is not available>, positional_params=0x40020000, invoke_invalid_param_handler=0x7f, pf_args=0x140020000, args_ctx=0x140020000, valist=0x7f84280910) in msvcrt (0x0000007f83e8fc50) 4 0x0000007f83d67ec0 MSVCRT_vsnwprintf+0x47(str=<is not available>, len=<is not available>, format=<is not available>, valist={__stack=0x7f836819f0, __gr_top=0x140010170, __vr_top=0x7f8428071c, __gr_offs=0x83681a28, __vr_offs=0x7f}) in msvcrt (0x0000007f83e8fc80) 5 0x0000000140012328 in makecat (+0x12327) (0x0000007f83e8fcb0) 6 0x00000001400123b0 in makecat (+0x123af) (0x0000007f83e8fcb0)
uuidgen.exe -i this generates a different path which involves MSVCRT_vsnprintf, pf_printf_a and pf_handle_string_a
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #1 from André H. nerv@dawncrow.de --- Created attachment 51821 --> https://bugs.winehq.org/attachment.cgi?id=51821 running makecat.exe
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #2 from André H. nerv@dawncrow.de --- Created attachment 51822 --> https://bugs.winehq.org/attachment.cgi?id=51822 running uuidgen.exe -i
https://bugs.winehq.org/show_bug.cgi?id=38886
André H. nerv@dawncrow.de changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |UPSTREAM
--- Comment #3 from André H. nerv@dawncrow.de --- resolving upstream
https://bugs.winehq.org/show_bug.cgi?id=38886
Martin Storsjö wine@martin.st changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |wine@martin.st
--- Comment #4 from Martin Storsjö wine@martin.st --- One attempt at implementing __builtin_ms_va_list for arm64, for clang, is in https://reviews.llvm.org/D34474 and https://reviews.llvm.org/D34475.
When using such a clang build, one has to add "-Xclang -target-feature -Xclang +reserve-x18" to the cflags, in order to avoid bug 38780.
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #5 from Martin Storsjö martin@martin.st --- This should be fixed in 8fb8cc03c3edb599dd98f369e14a08f899cbff95 (pushed on May 11th) now, requiring Clang 5.0 or newer, which supports __builtin_ms_va_list for aarch64.
https://bugs.winehq.org/show_bug.cgi?id=38886
André H. nerv@dawncrow.de changed:
What |Removed |Added ---------------------------------------------------------------------------- Fixed by SHA1| |8fb8cc03c3edb599dd98f369e14 | |a08f899cbff95 Resolution|NOTOURBUG |FIXED
--- Comment #6 from André H. nerv@dawncrow.de --- (In reply to Martin Storsjö from comment #5)
This should be fixed in 8fb8cc03c3edb599dd98f369e14a08f899cbff95 (pushed on May 11th) now, requiring Clang 5.0 or newer, which supports __builtin_ms_va_list for aarch64.
Fixed using clang
https://bugs.winehq.org/show_bug.cgi?id=38886
Alexandre Julliard julliard@winehq.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |CLOSED
--- Comment #7 from Alexandre Julliard julliard@winehq.org --- Closing bugs fixed in 3.9.
https://bugs.winehq.org/show_bug.cgi?id=38886
Michael Stefaniuc mstefani@winehq.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|--- |3.0.x
https://bugs.winehq.org/show_bug.cgi?id=38886
Michael Stefaniuc mstefani@winehq.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|3.0.x |---
--- Comment #8 from Michael Stefaniuc mstefani@winehq.org --- Removing the 3.0.x milestone from bugs included in 3.0.3.
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #9 from André H. nerv@dawncrow.de --- just for future reference, because it is not that easy to find all the pieces, the story goes:
Much earlier for x86_64 varargs ( https://bugs.llvm.org/show_bug.cgi?id=8851 ): { r186144:
https://github.com/llvm-mirror/llvm/commit/ac226bbf457f6b5e5210a4a82b1ce6782...
r189644:
https://github.com/llvm-mirror/clang/commit/e8519c31a6ef853b627d557702ac1890...
r245990:
https://github.com/llvm-mirror/llvm/commit/7e96f0f6fffbdebfdac238ae76cc3a791... discussions: http://reviews.llvm.org/D1622
r247941:
https://github.com/llvm-mirror/clang/commit/69b5694b76d36fde04c1f3511459a575... discussions: http://reviews.llvm.org/D1623 }
r308208: https://github.com/llvm-mirror/llvm/commit/6c132cb749932c66a27c9e19ea5426173... discussions: https://reviews.llvm.org/D34474
r308209: https://github.com/llvm-mirror/clang/commit/ebe8bde59376321af40abf168495ef8d...
r308218: https://github.com/llvm-mirror/clang/commit/ba419afb8473f35d5e1d3931c9e88911... discussions: https://reviews.llvm.org/D34475
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #10 from Martin Storsjö martin@martin.st --- Another pretty relevant commit:
r309744: https://github.com/llvm-mirror/llvm/commit/39bec211f05ca28c8b61d70fd5d9c030f... discussions: https://reviews.llvm.org/D35919
(The original implementation of AArch64/Win64 varargs in the previous linked commits was buggy for a lot cases, while this version has been in use without any known issues now for long over a year.)
https://bugs.winehq.org/show_bug.cgi?id=38886
Anastasius Focht focht@gmx.net changed:
What |Removed |Added ---------------------------------------------------------------------------- URL|http://go.microsoft.com/fwl |https://web.archive.org/web |ink/p/?LinkId=536682 |/20200915210324/http://down | |load.microsoft.com/download | |/6/3/B/63BADCE0-F2E6-44BD-B | |2F9-60F5F073038E/standalone | |sdk/SDKSETUP.EXE
https://bugs.winehq.org/show_bug.cgi?id=38886
Patrick Littlefighter1996@googlemail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |Littlefighter1996@googlemai | |l.com
--- Comment #11 from Patrick Littlefighter1996@googlemail.com --- I've tried building the version as of tag 7.21 for aarch64 for an OpenEmbedded distro.
When using the environment scripts to setup the toolchain, wine runs, the applications built with wine work, but while running an ARM64 binary built with MSVC, the program crashes for functions that use varargs. (Only the function "_snwprintf_s" has been tested).
(Using GCC as the compiler for wine).
How is the ARM64 port currently supposed to be built? Using Clang (from llvm-mingw) using a ARM64 Linux target fails currently at two major points.
* Configure will fail for the 64-bit compare-and-swap detection. (I've forced that to "none needed", although I don't know if that's the proper approach, as the GCC configuration determined that, I figured, that's probably the correct setting). * All functions that use va_start require __cdecl to be specified, as these have been redefined as long as clang is being used. clang complains about "error: '__builtin_ms_va_start' used in System V ABI function" throughout MANY occurences.
Furthermore, linking ntdll fails when building, referencing a missing built-in function. One that apparently comes from GCC (either InterlockedCompareExchange64 or InterlockedCompareExchange128 specified in rtl.o that's using the built-in GCC functions). Apparently the object was built using GCC, but winegcc invokes clang and lld, failing to link the built-in function.
So, I'm unsure if this bug has regressed or if that's just me failing to properly configure the project.
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #12 from Martin Storsjö martin@martin.st --- (In reply to Patrick from comment #11)
How is the ARM64 port currently supposed to be built? Using Clang (from llvm-mingw) using a ARM64 Linux target fails currently at two major points.
- Configure will fail for the 64-bit compare-and-swap detection.
(I've forced that to "none needed", although I don't know if that's the proper approach, as the GCC configuration determined that, I figured, that's probably the correct setting).
- All functions that use va_start require __cdecl to be specified, as these
have been redefined as long as clang is being used. clang complains about "error: '__builtin_ms_va_start' used in System V ABI function" throughout MANY occurences.
Furthermore, linking ntdll fails when building, referencing a missing built-in function. One that apparently comes from GCC (either InterlockedCompareExchange64 or InterlockedCompareExchange128 specified in rtl.o that's using the built-in GCC functions). Apparently the object was built using GCC, but winegcc invokes clang and lld, failing to link the built-in function.
So, I'm unsure if this bug has regressed or if that's just me failing to properly configure the project.
You need to provide more details about how you configure your build - I test building for ARM64 nightly, and it runs just fine, supporting vararg functions just fine. The two issues you mentioned sound entirely unfamiliar to me.
On Ubuntu 20.04, with llvm-mingw based on LLVM 15.0.0 (https://github.com/mstorsjo/llvm-mingw/releases/tag/20220906 - it's possible to use Ubuntu provided Clang too, but at least on 20.04, the bundled version is too old/has some bugs that are fixed in newer versions) added to $PATH, with wine-8.0-rc5:
CC="clang" ../wine/configure --prefix=$HOME/wine-test --disable-tests --without-freetype --without-x make make install export PATH=$HOME/wine-test:$PATH export WINEPREFIX=$HOME/.wine-test wine <test.exe>
It's also possible to build with GCC for the host bits, by leaving out CC="clang". The --disable-tests and --without-* bits are of course also optional, that's just how I just tested it. (wine-7.21 should also work just as well as 8.0-rc5 in general. There were some temporary regressions in my build tests around that date, but they shouldn't have affected the aarch64 builds.)
It's quite possible to cross compile it too, but that requires a couple more twists. (When cross compiling, it's essential that you first build the host side tools natively, for an exactly matching version of wine. For such builds, I first configure a native build with essentially just "../wine/configure" and do "make __tooldeps__ nls/all", then configure a regular autotools cross build of wine with "./configure --host=aarch64-linux-gnu --with-wine-tools=/home/martin/code/wine-build" (where the directory is the build directory for the native build where I built __tooldeps__).
(I'm not familiar with OpenEmbedded unfortunately so I can't give hints specific for that setup though.)
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #13 from Patrick Littlefighter1996@googlemail.com --- Thanks for the response. I'll try again with 8.0-rc5.
I'm indeed cross-compiling. Maybe that's the problem. I have already built the version of Wine 7.21 for my host machine and therefor have the wine-tools.
I'm using following configure command, after I've sourced the environment file:
../configure --enable-win64 --with-alsa CC="ccache /opt/tdx-xwayland/2.6.3/sysroots/x86_64-tdxsdk-linux/usr/bin/aarch64-tdx-linux/aarch64-tdx-linux-gcc --sysroot=/opt/tdx-xwayland/2.6.3/sysroots/aarch64-tdx-linux" CXX="ccache /opt/tdx-xwayland/2.6.3/sysroots/x86_64-tdxsdk-linux/usr/bin/aarch64-tdx-linux/aarch64-tdx-linux-gcc --sysroot=/opt/tdx-xwayland/2.6.3/sysroots/aarch64-tdx-linux" LD=/opt/tdx-xwayland/2.6.3/sysroots/x86_64-tdxsdk-linux/usr/bin/aarch64-tdx-linux/aarch64-tdx-linux-ld 'CFLAGS=-Og -gdwarf-4 --sysroot=/opt/tdx-xwayland/2.6.3/sysroots/aarch64-tdx-linux' 'CROSSCFLAGS=-Og -gdwarf-4' 'LDFLAGS=--sysroot=/opt/tdx-xwayland/2.6.3/sysroots/aarch64-tdx-linux -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed' --host=aarch64-tdx-linux --with-wine-tools=../build-g-7.21 FREETYPE_CFLAGS=-I/opt/tdx-xwayland/2.6.3/sysroots/aarch64-tdx-linux/usr/include/freetype2 --without-gnutls --prefix=/opt/tdx-xwayland/2.6.3/wine --disable-win16 --disable-wow64 --disable-wow64win --disable-wow64cpu
GCC: aarch64-tdx-linux-gcc (GCC) 8.2.0 Clang: clang version 15.0.0 (4ba6a9c9f65bbc8bd06e3652cb20fd4dfc846137)
I've noticed that specifying the "--sysroot" parameter in CROSSCFLAGS as well breaks the build completely in my case. (Binary builds, but can't load DLLs, strings are corrupted. e.g tries to load "n.drv" instead of "winex11.drv")
So if I understood correctly, using gcc (for CC) AND llvm-mingw's clang through winegcc (for "CROSSCC"), as well as only using clang the build should actually work? If that's the case, I would try it again by emulating the target machine, setup a toolchain and try to run the build there. (After all, if clang works for you, it sounds like that's an issue with cross-compiling).
I assume there is a test that is being run to ensure that varargs don't break? If yes, maybe it only fails under certain circumstances that aren't covered, yet? I would like to run it, to determine if it passes or fails on my build.
(my program runs only: "_snwprintf_s(buffer, _countof(buffer), _countof(buffer) - 1, L"%Ts + %Ts", L"A", L"B");" and crashes)
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #14 from Martin Storsjö martin@martin.st --- (In reply to Patrick from comment #13)
I'm using following configure command, after I've sourced the environment file:
../configure --enable-win64 --with-alsa CC="ccache /opt/tdx-xwayland/2.6.3/sysroots/x86_64-tdxsdk-linux/usr/bin/aarch64-tdx- linux/aarch64-tdx-linux-gcc --sysroot=/opt/tdx-xwayland/2.6.3/sysroots/aarch64-tdx-linux" CXX="ccache /opt/tdx-xwayland/2.6.3/sysroots/x86_64-tdxsdk-linux/usr/bin/aarch64-tdx- linux/aarch64-tdx-linux-gcc --sysroot=/opt/tdx-xwayland/2.6.3/sysroots/aarch64-tdx-linux" LD=/opt/tdx-xwayland/2.6.3/sysroots/x86_64-tdxsdk-linux/usr/bin/aarch64-tdx- linux/aarch64-tdx-linux-ld 'CFLAGS=-Og -gdwarf-4 --sysroot=/opt/tdx-xwayland/2.6.3/sysroots/aarch64-tdx-linux' 'CROSSCFLAGS=-Og -gdwarf-4' 'LDFLAGS=--sysroot=/opt/tdx-xwayland/2.6.3/sysroots/aarch64-tdx-linux -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed' --host=aarch64-tdx-linux --with-wine-tools=../build-g-7.21 FREETYPE_CFLAGS=-I/opt/tdx-xwayland/2.6.3/sysroots/aarch64-tdx-linux/usr/ include/freetype2 --without-gnutls --prefix=/opt/tdx-xwayland/2.6.3/wine --disable-win16 --disable-wow64 --disable-wow64win --disable-wow64cpu
GCC: aarch64-tdx-linux-gcc (GCC) 8.2.0 Clang: clang version 15.0.0 (4ba6a9c9f65bbc8bd06e3652cb20fd4dfc846137)
I've noticed that specifying the "--sysroot" parameter in CROSSCFLAGS as well breaks the build completely in my case. (Binary builds, but can't load DLLs, strings are corrupted. e.g tries to load "n.drv" instead of "winex11.drv")
Currently, the Wine build is done by building some parts with the regular aarch64-linux cross compiler (which can be either GCC or Clang), and some parts with an aarch64-pe targeting compiler (which only is supported by Clang at the moment). For other targets, it's possible to build everything with the linux-targeting compiler, which used to be the traditional way Wine is set up, but for aarch64 this configuration is no longer supported.
For the PE bits, if nothing is specified, iirc the configure script first looks for "aarch64-w64-mingw32-clang" and if not found, it checks if "clang --target=aarch64-windows" works; both run mostly the same code, but there are vague differences for how things are linked.
I presume that if you include that parameter in CROSSCFLAGS, it ends up used for building/linking the PE parts of the build too, and that's probably not desired at all.
So if I understood correctly, using gcc (for CC) AND llvm-mingw's clang through winegcc (for "CROSSCC"), as well as only using clang the build should actually work?
Yes
If that's the case, I would try it again by emulating the target machine, setup a toolchain and try to run the build there. (After all, if clang works for you, it sounds like that's an issue with cross-compiling).
Well - although cross compiling Wine from x86 to aarch64 also is supported. I don't test that quite nightly though, but I do test it semi-regularly manually.
I assume there is a test that is being run to ensure that varargs don't break?
I'm not aware of one specific test for that, but I do test running loads of various binaries that do call vararg functions, so I'm pretty sure that it's covered and working in my builds.
If yes, maybe it only fails under certain circumstances that aren't covered, yet? I would like to run it, to determine if it passes or fails on my build.
(my program runs only: "_snwprintf_s(buffer, _countof(buffer), _countof(buffer) - 1, L"%Ts + %Ts", L"A", L"B");" and crashes)
If you link to a specific binary containing this call, I can double check running that executable with my Wine builds.
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #15 from Patrick Littlefighter1996@googlemail.com --- Oh, that's great! Thank you.
I'm attaching the binary that crashes.
(It creates a window which title should read "A + B").
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #16 from Patrick Littlefighter1996@googlemail.com --- Created attachment 73928 --> https://bugs.winehq.org/attachment.cgi?id=73928 ARM64 program that uses _snwprintf_s to set the window title
https://bugs.winehq.org/show_bug.cgi?id=38886
Patrick Littlefighter1996@googlemail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Attachment #73928|0 |1 is obsolete| |
--- Comment #17 from Patrick Littlefighter1996@googlemail.com --- Created attachment 73929 --> https://bugs.winehq.org/attachment.cgi?id=73929 ARM64 console program that tests if _snwprintf_s works as expected
ARM64 program that uses _snwprintf_s to concatenate "A" and "B" and returns if result is as expected.
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #18 from Martin Storsjö martin@martin.st --- (In reply to Patrick from comment #17)
Created attachment 73929 [details] ARM64 console program that tests if _snwprintf_s works as expected
ARM64 program that uses _snwprintf_s to concatenate "A" and "B" and returns if result is as expected.
Thanks - this does print "Test Successful, strings equal" for me. (Thanks for providing the console program - most of my builds are with --without-x since I only run it headless.)
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #19 from Patrick Littlefighter1996@googlemail.com --- You're welcome!
I'm sorry, I noticed a mistake I made. I forgot to check for "==0" and also had a typo so it wasn't visible it was misbehaving. I'm updating the file, again.
But it already sounds promising, it isn't crashing.
https://bugs.winehq.org/show_bug.cgi?id=38886
Patrick Littlefighter1996@googlemail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Attachment #73929|0 |1 is obsolete| |
--- Comment #20 from Patrick Littlefighter1996@googlemail.com --- Created attachment 73930 --> https://bugs.winehq.org/attachment.cgi?id=73930 ARM64 console program that tests if _snwprintf_s works as expected
This one properly compares the strings.
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #21 from Martin Storsjö martin@martin.st --- (In reply to Patrick from comment #20)
Created attachment 73930 [details] ARM64 console program that tests if _snwprintf_s works as expected
This one properly compares the strings.
This one also runs successfully.
https://bugs.winehq.org/show_bug.cgi?id=38886
--- Comment #22 from Patrick Littlefighter1996@googlemail.com --- I've used the target system to build wine and now it's working as intended. I've noticed, all DLLs appear to be wrappers in the broken build (they are usually about 2550 bytes), while in the working build, it's the other way around.
On Ubuntu (through WSL) such a similar build (with small DLLs, probably wrappers around .so files), works just fine.
I suspect that this other (build-related) issue somehow manifests as this (varargs) issue.
I'm unsure if I should open an issue about this or not. Sound more like something that would belong in forums, if it was an issue specific to my toolchain.
-
wine-bugs＠winehq.org -
WineHQ Bugzilla