https://bugs.winehq.org/show_bug.cgi?id=46079
Bug ID: 46079 Summary: Support 32-bit ARM hard-float cross-toolchains with GNU binutils assembler that doesn't implicitly assume '.fpu {vfp,neon}' (Yocto/Poky SDK) Product: Wine Version: 3.19 Hardware: x86-64 OS: Linux Status: NEW Severity: normal Priority: P2 Component: tools Assignee: wine-bugs@winehq.org Reporter: focht@gmx.net Distribution: ---
Hello folks,
after working around bug 46053 ("Support cross-toolchains that need target sysroot explicitly passed via ' --sysroot' command-line option (Yocto/Poky SDK)") by having 'winegcc' passing the target sysroot to the cross-compiler/linker via '--sysroot', aarch64 Wine now cross compiles successfully with a full target sysroot.
-> aarch64-poky-linux
------------------------------------------------------------------------- Build Configuration: BB_VERSION = "1.38.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "fedora-28" TARGET_SYS = "aarch64-poky-linux" MACHINE = "qemuarm64" DISTRO = "poky" DISTRO_VERSION = "2.5.1" TUNE_FEATURES = "aarch64" TARGET_FPU = "" meta meta-poky meta-yocto-bsp = "sumo:d240b885f26e9b05c8db0364ab2ace9796709aad" meta-oe = "sumo:2bb21ef27c4b0c9d52d30b3b2c5a0160fd02b966" meta-clang = "sumo:1784b06e6b1a83748438b4bec7e079d735d5b608" ... -------------------------------------------------------------------------
Building for 32-bit ARMv7 hard-float fails (using the same mechanics).
-> arm-poky-linux-gnueabi (cortexa9hf-vfpv3-poky-linux-gnueabi)
------------------------------------------------------------------------- Build Configuration: BB_VERSION = "1.38.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "fedora-28" TARGET_SYS = "arm-poky-linux-gnueabi" MACHINE = "qemuarma9" DISTRO = "poky" DISTRO_VERSION = "2.5.1" TUNE_FEATURES = "arm armv7a vfp vfpv3d16 vfpv3 callconvention-hard cortexa9" TARGET_FPU = "hard" meta meta-poky meta-yocto-bsp = "sumo:d240b885f26e9b05c8db0364ab2ace9796709aad" meta-oe = "sumo:2bb21ef27c4b0c9d52d30b3b2c5a0160fd02b966" meta-clang = "sumo:1784b06e6b1a83748438b4bec7e079d735d5b608" ... -------------------------------------------------------------------------
--- snip --- ... /home/focht/projects/wine/mainline-build-x86_64/tools/winegcc/winegcc -o d2d1.dll.so \ -B/home/focht/projects/wine/mainline-build-x86_64/tools/winebuild --sysroot=../.. -b \ arm-poky-linux-gnueabi -marm -mfloat-abi=hard -fasynchronous-unwind-tables -shared /home/focht/projects/wine/mainline-src/dlls/d2d1/d2d1.spec \ bitmap.o bitmap_render_target.o brush.o dc_render_target.o device.o effect.o factory.o geometry.o \ hwnd_render_target.o layer.o mesh.o state_block.o stroke.o wic_render_target.o version.res \ -ldwrite -ld3d10_1 ../../dlls/dxguid/libdxguid.a ../../dlls/uuid/libuuid.a -lgdi32 -luser32 \ -ladvapi32 ../../libs/port/libwine_port.a -Wb,-ddwrite -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed d2d1.IYX82U.s: Assembler messages: d2d1.IYX82U.s:291: Error: selected processor does not support `vpush {s0-s15}' in ARM mode d2d1.IYX82U.s:310: Error: selected processor does not support `vpush {s0-s15}' in ARM mode winebuild: arm-poky-linux-gnueabi-gcc failed with status 1 winegcc: /home/focht/projects/wine/mainline-build-x86_64/tools/winebuild/winebuild failed make[1]: *** [Makefile:1440: d2d1.dll.so] Error 2 make[1]: Leaving directory '/home/focht/projects/wine/mainline-build-arm/dlls/d2d1' make: *** [Makefile:8385: dlls/d2d1] Error 2 --- snip ---
Re-executing failing last command with '-v -save-temps':
--- snip --- /home/focht/projects/wine/mainline-build-x86_64/tools/winegcc/winegcc -v -save-temps -o d2d1.dll.so -B/home/focht/projects/wine/mainline-build-x86_64/tools/winebuild --sysroot=../.. -b arm-poky-linux-gnueabi -marm -mfloat-abi=hard -fasynchronous-unwind-tables -shared /home/focht/projects/wine/mainline-src/dlls/d2d1/d2d1.spec bitmap.o bitmap_render_target.o brush.o dc_render_target.o device.o effect.o factory.o geometry.o hwnd_render_target.o layer.o mesh.o state_block.o stroke.o wic_render_target.o version.res -ldwrite -ld3d10_1 ../../dlls/dxguid/libdxguid.a ../../dlls/uuid/libuuid.a -lgdi32 -luser32 -ladvapi32 ../../libs/port/libwine_port.a -Wb,-ddwrite -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed /home/focht/projects/wine/mainline-build-x86_64/tools/winebuild/winebuild -v --save-temps --target arm-poky-linux-gnueabi -fasynchronous-unwind-tables --cc-cmd=arm-poky-linux-gnueabi-gcc --ld-cmd=arm-poky-linux-gnueabi-ld -D_REENTRANT -fPIC --dll -o d2d1.dll-IpMxH8.spec.o -E /home/focht/projects/wine/mainline-src/dlls/d2d1/d2d1.spec -L../../dlls -L../../libs/wine -marm -mfloat-abi=hard -ddwrite -rversion.res -- bitmap.o bitmap_render_target.o brush.o dc_render_target.o device.o effect.o factory.o geometry.o hwnd_render_target.o layer.o mesh.o state_block.o stroke.o wic_render_target.o ../../dlls/dwrite/libdwrite.def ../../dlls/d3d10_1/libd3d10_1.def ../../dlls/dxguid/libdxguid.a ../../dlls/uuid/libuuid.a ../../dlls/gdi32/libgdi32.def ../../dlls/user32/libuser32.def ../../dlls/advapi32/libadvapi32.def ../../libs/port/libwine_port.a ../../dlls/winecrt0/libwinecrt0.a ../../dlls/kernel32/libkernel32.def ../../dlls/ntdll/libntdll.def arm-poky-linux-gnueabi-gcc -xassembler -c -o d2d1.nY1v31.o d2d1.CJoop5.s arm-poky-linux-gnueabi-ld -r -o d2d1.KKS2kZ.o d2d1.nY1v31.o bitmap.o bitmap_render_target.o brush.o dc_render_target.o device.o effect.o factory.o geometry.o hwnd_render_target.o layer.o mesh.o state_block.o stroke.o wic_render_target.o ../../dlls/dxguid/libdxguid.a ../../dlls/uuid/libuuid.a ../../libs/port/libwine_port.a ../../dlls/winecrt0/libwinecrt0.a arm-poky-linux-gnueabi-gcc -xassembler -c -o d2d1.dll-IpMxH8.spec.o d2d1.vn7wK8.s d2d1.vn7wK8.s: Assembler messages: d2d1.vn7wK8.s:291: Error: selected processor does not support `vpush {s0-s15}' in ARM mode d2d1.vn7wK8.s:310: Error: selected processor does not support `vpush {s0-s15}' in ARM mode winebuild: arm-poky-linux-gnueabi-gcc failed with status 1 winegcc: /home/focht/projects/wine/mainline-build-x86_64/tools/winebuild/winebuild failed --- snip ---
$ cat d2d1.vn7wK8.s /* File generated automatically from /home/focht/projects/wine/mainline-src/dlls/d2d1/d2d1.spec; do not edit! */ /* This file can be copied, modified and distributed without restriction. */
.section ".text","ax" b 1f __wine_spec_pe_header: .skip 69632 1:
.data .align 2 .globl __wine_spec_nt_header .hidden __wine_spec_nt_header __wine_spec_nt_header: .L__wine_spec_rva_base: ... .L__wine_spec_exports_funcs: .long D2D1CreateFactory .long D2D1MakeRotateMatrix .long D2D1MakeSkewMatrix .long D2D1IsMatrixInvertible .long D2D1InvertMatrix .long __wine_stub_D2D1ConvertColorSpace .long __wine_stub_D2D1CreateDevice .long __wine_stub_D2D1CreateDeviceContext .long __wine_stub_D2D1SinCos .long __wine_stub_D2D1Tan .long __wine_stub_D2D1Vec3Length ... __wine_spec_relay_entry_points: nop .align 2 .L__wine_spec_relay_entry_point_1: .cfi_startproc push {r0-r3} mov r2, SP push {LR} sub SP, #4 mov r1,#393216 ldr r0, 2f add r0, PC ldr IP, [r0, #4] 1: blx IP ldr IP, [SP, #4] add SP, #24 bx IP 2: .long .L__wine_spec_relay_descr-1b .cfi_endproc .align 2 .L__wine_spec_relay_entry_point_2: .cfi_startproc push {r0-r3} mov r2, SP vpush {s0-s15} push {LR} sub SP, #4 mov r1,#1 add r1,#65536 ldr r0, 2f add r0, PC ldr IP, [r0, #4] 1: blx IP ldr IP, [SP, #4] add SP, #88 bx IP 2: .long .L__wine_spec_relay_descr-1b .cfi_endproc .align 2 --- snip ---
https://source.winehq.org/git/wine.git/blob/HEAD:/tools/winebuild/spec32.c#l...
--- snip --- ... case CPU_ARM: { unsigned int mask, val, count = 0; int j, has_float = 0;
if (strcmp( float_abi_option, "soft" )) for (j = 0; j < odp->u.func.nb_args && !has_float; j++) has_float = is_float_arg( odp, j );
val = (odp->u.func.args_str_offset << 16) | (i - spec->base); output( "\tpush {r0-r3}\n" ); output( "\tmov r2, SP\n"); if (has_float) output( "\tvpush {s0-s15}\n" ); output( "\tpush {LR}\n" ); output( "\tsub SP, #4\n"); for (mask = 0xff; mask; mask <<= 8) if (val & mask) output( "\t%s r1,#%u\n", count++ ? "add" : "mov", val & mask ); if (!count) output( "\tmov r1,#0\n" ); output( "\tldr r0, 2f\n"); output( "\tadd r0, PC\n"); output( "\tldr IP, [r0, #4]\n"); output( "1:\tblx IP\n"); output( "\tldr IP, [SP, #4]\n" ); output( "\tadd SP, #%u\n", 24 + (has_float ? 64 : 0) ); output( "\tbx IP\n"); output( "2:\t.long .L__wine_spec_relay_descr-1b\n" ); break; ... --- snip ---
The code was introduced by commit https://source.winehq.org/git/wine.git/commitdiff/5e3534ee411a8c5baea5022705... ("ntdll: Fix handling of floating point arguments in relay debugging on ARM.")
The problem is that Wine makes the assumption that the assembler from hardfloat binutils implicitly assumes '.fpu {vfp,neon}' without any command line options. This isn't necessarily true for most hard-float cross-toolchains but for native toolchains, running on actual targets. Usually this is solved by the cross-toolchain environment by passing the proper '-mfloat-abi=' and/or '-mfpu=' options when cross-compiling.
Simple test with .c file:
--- snip --- $ cat vfptest.c void foo(void){ __asm__ volatile("fadds s0, s0, s0"); } --- snip ---
--- snip --- $ arm-poky-linux-gnueabi-gcc -c vfptest.c /tmp/cc3DpeRI.s: Assembler messages: /tmp/cc3DpeRI.s:27: Error: selected processor does not support `fadds s0,s0,s0' in ARM mode
$ arm-poky-linux-gnueabi-gcc -mfloat-abi=hard -c vfptest.c --- snip ---
Test to show the available fpu target options:
--- snip --- $ arm-poky-linux-gnueabi-gcc -mfpu=? arm-poky-linux-gnueabi-gcc: error: unrecognized argument in option ‘-mfpu=?’
arm-poky-linux-gnueabi-gcc: note: valid arguments to ‘-mfpu=’ are: auto crypto-neon-fp-armv8 fp-armv8 fpv4-sp-d16 fpv5-d16 fpv5-sp-d16 neon neon-fp-armv8 neon-fp16 neon-vfpv3 neon-vfpv4 vfp vfp3 vfpv2 vfpv3 vfpv3-d16 vfpv3-d16-fp16 vfpv3-fp16 vfpv3xd vfpv3xd-fp16 vfpv4 vfpv4-d16
arm-poky-linux-gnueabi-gcc: fatal error: no input files compilation terminated. --- snip ---
Simple test with assembly file, illustrating the problem with Wine:
--- snip --- $ cat vfptest.S .text test: fadds s0, s0, s0 --- snip ---
NOTE: 'vfp' is an alias for 'vfpv2', 'neon' is an alias for 'neon-vfpv3'
--- snip --- $ arm-poky-linux-gnueabi-gcc -c vfptest.s vfptest.s: Assembler messages: vfptest.s:3: Error: selected processor does not support `fadds s0,s0,s0' in ARM mode
$ arm-poky-linux-gnueabi-gcc -c -mfloat-abi=hard vfptest.s vfptest.s: Assembler messages: vfptest.s:3: Error: selected processor does not support `fadds s0,s0,s0' in ARM mode
$ arm-poky-linux-gnueabi-gcc -c -mfpu=vfp vfptest.s $ arm-poky-linux-gnueabi-gcc -c -mfpu=vfpv2 vfptest.s $ arm-poky-linux-gnueabi-gcc -c -mfpu=vfpv3 vfptest.s $ arm-poky-linux-gnueabi-gcc -c -mfpu=vfpv4 vfptest.s $ arm-poky-linux-gnueabi-gcc -c -mfpu=neon vfptest.s ... --- snip ---
Unlike using C with inline assembly, real assembly sources need either '.fpu' directive, or '-mfpu=' option provided in the command line.
Using CFLAGS, EXTRACFLAGS to set '-mfpu=' doesn't work because of the 'winegcc' wrapper limitation (like bug 46053 it builds a custom command line, not passing these option down).
Forcing to a use hardfloat binutils with assembler that implicitly assumes '.fpu vfp' without any command line options is IMHO not feasible. I don't want to change/hack upstream toolchain projects.
Changing 'winebuild' to add explicit '.fpu vfp' or '.fpu neon' directives to the assembly sources in case of 32-bit ARM hard-float seems reasonable to me.
https://source.winehq.org/git/wine.git/blob/HEAD:/tools/winebuild/spec32.c
Simple test:
--- snip --- $ cat vfptest.s .fpu vfp .text test: fadds s0, s0, s0 --- snip ---
--- snip --- $ arm-poky-linux-gnueabi-gcc -c vfptest.s --- snip ---
With that problem fixed, Wine now cross compiles successfully with a full 32-bit ARM hardfloat target sysroot (same x11/freetype/libxml2 etc. deps as for my x86_64 host).
Next step would be running apps/tests on my x86_64 host with 'qemuarm64' and 'qemuarma9' using the generic ARM system emulation with the 'virt machine' machine which also supports 'virtio-gpu' for full graphics stack.
$ wine --version wine-3.19-117-g4852130c82
Regards