[PATCH 0/5] MR5884: Use zerofill sections instead of preloader on macOS when building with Xcode 15.3.
Xcode 15.3 adds a new linker flag (`-no_huge`) which allows the loader to use zero-fill sections to reserve the areas currently being reserved by the preloader. This means the preloader is no longer needed (a good thing, since it's heavily dependent on private APIs). The preloader will still be used when Xcode <15.3 is being used, or when building for i386 (32-bit for 10.14 and earlier). I've also tested a 64-bit wineloader built with Xcode 15.3 on macOS 10.13 and found that it works correctly. The only user-visible change I've found is that `setprogname()` now correctly changes the process name, so a `ps` listing will now show (for example) `C:\windows\regedit.exe` (as it does on Linux) instead of `/Users/bshanks/wine/build64/loader/wine-preloader C:\windows\regedit.exe`. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/5884
From: Brendan Shanks <bshanks(a)codeweavers.com> This is already the default for all Mac binaries targeting 10.7 or later. --- configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index 231ec7f17d6..f7e27f591a0 100644 --- a/configure.ac +++ b/configure.ac @@ -637,7 +637,7 @@ case $host_os in AC_SUBST(SECURITY_LIBS,"-framework Security -framework CoreFoundation") AC_SUBST(SYSTEMCONFIGURATION_LIBS,"-framework SystemConfiguration") - WINELOADER_LDFLAGS="-Wl,-pie,-segalign,0x1000,-pagezero_size,0x1000,-sectcreate,__TEXT,__info_plist,loader/wine_info.plist" + WINELOADER_LDFLAGS="-Wl,-segalign,0x1000,-pagezero_size,0x1000,-sectcreate,__TEXT,__info_plist,loader/wine_info.plist" case $HOST_ARCH in i386|x86_64) wine_can_build_preloader=yes ;; -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5884
From: Brendan Shanks <bshanks(a)codeweavers.com> --- configure.ac | 2 -- 1 file changed, 2 deletions(-) diff --git a/configure.ac b/configure.ac index f7e27f591a0..b88b70d4a55 100644 --- a/configure.ac +++ b/configure.ac @@ -659,8 +659,6 @@ case $host_os in esac dnl If preloader is used, the loader needs to be an LC_UNIXTHREAD binary to avoid AppKit/Core Animation problems. WINELOADER_LDFLAGS="$WINELOADER_LDFLAGS -mmacosx-version-min=10.7" - else - WINE_WARNING([can't build Wine preloader; many programs won't work]) fi if test "x$with_coreaudio" != "xno"; -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5884
From: Brendan Shanks <bshanks(a)codeweavers.com> --- configure.ac | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/configure.ac b/configure.ac index b88b70d4a55..db0c88a04f4 100644 --- a/configure.ac +++ b/configure.ac @@ -640,11 +640,11 @@ case $host_os in WINELOADER_LDFLAGS="-Wl,-segalign,0x1000,-pagezero_size,0x1000,-sectcreate,__TEXT,__info_plist,loader/wine_info.plist" case $HOST_ARCH in - i386|x86_64) wine_can_build_preloader=yes ;; - *) wine_can_build_preloader=no ;; + i386|x86_64) wine_use_preloader=yes ;; + *) wine_use_preloader=no ;; esac - if test "$wine_can_build_preloader" = "yes" + if test "$wine_use_preloader" = "yes" then WINEPRELOADER_LDFLAGS="-nostartfiles -nodefaultlibs -e _start -ldylib1.o -mmacosx-version-min=10.7 -Wl,-no_new_main,-segalign,0x1000,-pagezero_size,0x1000,-sectcreate,__TEXT,__info_plist,loader/wine_info.plist" WINE_TRY_CFLAGS([-Wl,-no_pie], @@ -2033,19 +2033,17 @@ case $host_os in linux*) if test $HOST_ARCH != unknown then - test "$wine_binary" = wine || WINE_IGNORE_FILE(loader/wine-preloader) - WINELOADER_PROGRAMS="$WINELOADER_PROGRAMS $wine_binary-preloader" - fi - ;; - darwin*|macosx*) - if test "$wine_can_build_preloader" = "yes" - then - test "$wine_binary" = wine || WINE_IGNORE_FILE(loader/wine-preloader) - WINELOADER_PROGRAMS="$WINELOADER_PROGRAMS $wine_binary-preloader" + wine_use_preloader=yes fi ;; esac +if test "$wine_use_preloader" = "yes" +then + test "$wine_binary" = wine || WINE_IGNORE_FILE(loader/wine-preloader) + WINELOADER_PROGRAMS="$WINELOADER_PROGRAMS $wine_binary-preloader" +fi + dnl **** Check for functions **** dnl Check for -ldl -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5884
From: Brendan Shanks <bshanks(a)codeweavers.com> --- configure.ac | 1 + dlls/ntdll/unix/loader.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/configure.ac b/configure.ac index db0c88a04f4..1ed6bf059ea 100644 --- a/configure.ac +++ b/configure.ac @@ -2042,6 +2042,7 @@ if test "$wine_use_preloader" = "yes" then test "$wine_binary" = wine || WINE_IGNORE_FILE(loader/wine-preloader) WINELOADER_PROGRAMS="$WINELOADER_PROGRAMS $wine_binary-preloader" + AC_DEFINE(USE_PRELOADER, 1, [Define to 1 if the preloader is being used.]) fi dnl **** Check for functions **** diff --git a/dlls/ntdll/unix/loader.c b/dlls/ntdll/unix/loader.c index 8cf6abed3da..b97954a8182 100644 --- a/dlls/ntdll/unix/loader.c +++ b/dlls/ntdll/unix/loader.c @@ -147,7 +147,7 @@ SYSTEM_SERVICE_TABLE KeServiceDescriptorTable[4] = static void fatal_error( const char *err, ... ) __attribute__((noreturn, format(printf,1,2))); #endif -#if defined(linux) || defined(__APPLE__) +#ifdef USE_PRELOADER static const BOOL use_preloader = TRUE; #else static const BOOL use_preloader = FALSE; @@ -2074,7 +2074,7 @@ static int pre_exec(void) return 1; /* we have a preloader on x86-64/arm64 */ } -#elif defined(__APPLE__) && (defined(__i386__) || defined(__x86_64__)) +#elif defined(__APPLE__) && defined(USE_PRELOADER) static int pre_exec(void) { -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5884
From: Brendan Shanks <bshanks(a)codeweavers.com> Xcode 15.3 adds a new linker flag ('-no_huge') which allows the loader to use zero-fill sections to reserve the areas currently being reserved by the preloader. This means the preloader is no longer needed (a good thing, since it's heavily dependent on private APIs). The preloader will still be used when Xcode <15.3 is being used, or when building for i386 (32-bit for 10.14 and earlier). --- configure.ac | 11 +++++++++-- loader/main.c | 45 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+), 2 deletions(-) diff --git a/configure.ac b/configure.ac index 1ed6bf059ea..ccd0932a37e 100644 --- a/configure.ac +++ b/configure.ac @@ -640,8 +640,15 @@ case $host_os in WINELOADER_LDFLAGS="-Wl,-segalign,0x1000,-pagezero_size,0x1000,-sectcreate,__TEXT,__info_plist,loader/wine_info.plist" case $HOST_ARCH in - i386|x86_64) wine_use_preloader=yes ;; - *) wine_use_preloader=no ;; + i386) wine_use_preloader=yes ;; + x86_64) + dnl If the -no_huge linker option is present (added in Xcode 15.3), use zerofill sections instead of the preloader + WINE_TRY_CFLAGS([-Wl,-no_huge], + [wine_use_preloader=no + WINELOADER_LDFLAGS="$WINELOADER_LDFLAGS -Wl,-no_pie,-image_base,0x200000000,-no_huge,-no_fixup_chains,-segalign,0x1000,-segaddr,WINE_RESERVE,0x1000,-segaddr,WINE_TOP_DOWN,0x7ff000000000"], + [wine_use_preloader=yes]) + ;; + *) wine_use_preloader=no ;; esac if test "$wine_use_preloader" = "yes" diff --git a/loader/main.c b/loader/main.c index c258e025183..8778916623f 100644 --- a/loader/main.c +++ b/loader/main.c @@ -25,6 +25,7 @@ #include <stdio.h> #include <stdlib.h> #include <string.h> +#include <sys/mman.h> #include <sys/types.h> #include <sys/stat.h> #include <unistd.h> @@ -38,9 +39,51 @@ extern char **environ; +#if defined(__APPLE__) && defined(__x86_64__) && !defined(USE_PRELOADER) + +/* Not using the preloader on x86_64: + * Reserve the same areas as the preloader does, but using zero-fill sections + * (the only way to prevent system frameworks from using them, including allocations + * before main() runs). + */ +__asm__(".zerofill WINE_RESERVE,WINE_RESERVE"); +static char __wine_reserve[0x1fffff000] __attribute__((section("WINE_RESERVE, WINE_RESERVE"))); + +__asm__(".zerofill WINE_TOP_DOWN,WINE_TOP_DOWN"); +static char __wine_top_down[0x001ff0000] __attribute__((section("WINE_TOP_DOWN, WINE_TOP_DOWN"))); + +static const struct wine_preload_info preload_info[] = +{ + { __wine_reserve, sizeof(__wine_reserve) }, /* 0x1000 - 0x200000000: low 8GB */ + { __wine_top_down, sizeof(__wine_top_down) }, /* 0x7ff000000000 - 0x7ff001ff0000: top-down allocations + virtual heap */ + { 0, 0 } /* end of list */ +}; + +const __attribute((visibility("default"))) struct wine_preload_info *wine_main_preload_info = preload_info; + +static void init_reserved_areas(void) +{ + int i; + + for (i = 0; wine_main_preload_info[i].size != 0; i++) + { + /* Match how the preloader maps reserved areas: */ + mmap(wine_main_preload_info[i].addr, wine_main_preload_info[i].size, PROT_NONE, + MAP_FIXED | MAP_NORESERVE | MAP_PRIVATE | MAP_ANON, -1, 0); + } +} + +#else + /* the preloader will set this variable */ const __attribute((visibility("default"))) struct wine_preload_info *wine_main_preload_info = NULL; +static void init_reserved_areas(void) +{ +} + +#endif + /* canonicalize path and return its directory name */ static char *realpath_dirname( const char *name ) { @@ -177,6 +220,8 @@ int main( int argc, char *argv[] ) { void *handle; + init_reserved_areas(); + if ((handle = load_ntdll( argv[0] ))) { void (*init_func)(int, char **, char **) = dlsym( handle, "__wine_main" ); -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5884
Hi, It looks like your patch introduced the new failures shown below. Please investigate and fix them before resubmitting your patch. If they are not new, fixing them anyway would help a lot. Otherwise please ask for the known failures list to be updated. The tests also ran into some preexisting test failures. If you know how to fix them that would be helpful. See the TestBot job for the details: The full results can be found at: https://testbot.winehq.org/JobDetails.pl?Key=146424 Your paranoid android. === debian11b (64 bit WoW report) === dxgi: dxgi.c:3334: Test failed: Got monitor rect (0,0)-(1280,1024), expected (0,0)-(1024,768). dxgi.c:3336: Test failed: Got monitor rect (0,0)-(1280,1024), expected (0,0)-(1024,768). dxgi.c:3349: Test failed: Got monitor rect (0,0)-(1280,1024), expected (0,0)-(1024,768). dxgi.c:7387: Test failed: Got a different mode. dxgi.c:7390: Test failed: Got a different mode. dxgi.c:7452: Test failed: Got a different mode.
I've applied this patch locally and the resulting `wine` binary will segfault. ``` zsh: segmentation fault wine explorer ``` If I set `ac_cv_cflags__Wl__no_huge=no` that build works without issue. <br> ``` macOS 14.5 23F79 arm64 Command Line Tools 15.3.0.0.1.1708646388 ``` -- https://gitlab.winehq.org/wine/wine/-/merge_requests/5884#note_73841
On Thu Jun 20 20:51:09 2024 +0000, Dean M Greer wrote:
I've applied this patch locally and the resulting `wine` binary will segfault. ``` zsh: segmentation fault wine explorer ``` If I set `ac_cv_cflags__Wl__no_huge=no` that build works without issue. <br> ``` macOS 14.5 23F79 arm64 Command Line Tools 15.3.0.0.1.1708646388 ``` EDIT: I should add I'm building for x86_64, adding to avoid any possible confusion. Hmm, are you running from the build directory or installed? (Both cases seem to be working for me, just curious). Can you run it in lldb and get a backtrace, or is there a crash log in `~/Library/Logs/DiagnosticReports` or `/Library/Logs/DiagnosticReports`?
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/5884#note_73842
On Thu Jun 20 20:51:09 2024 +0000, Brendan Shanks wrote:
Hmm, are you running from the build directory or installed? (Both cases seem to be working for me, just curious). Can you run it in lldb and get a backtrace, or is there a crash log in `~/Library/Logs/DiagnosticReports` or `/Library/Logs/DiagnosticReports`? [wine-2024-06-20-165643.ips](/uploads/712225709eb587cc83a7a9373a9d2eb2/wine-2024-06-20-165643.ips)
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/5884#note_73843
On Thu Jun 20 20:57:48 2024 +0000, Dean M Greer wrote:
Running from install directory. [wine-2024-06-20-165643.ips](/uploads/712225709eb587cc83a7a9373a9d2eb2/wine-2024-06-20-165643.ips) Do you have any other hacks/patches applied?
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/5884#note_73844
On Thu Jun 20 21:09:42 2024 +0000, Brendan Shanks wrote:
Do you have any other hacks/patches applied? The applied patches are the ones found https://github.com/macports/macports-ports/tree/738dcbe7d0e425149d42102465d8...
Plus MR 5871 -- https://gitlab.winehq.org/wine/wine/-/merge_requests/5884#note_73845
On Thu Jun 20 21:18:17 2024 +0000, Dean M Greer wrote:
https://github.com/macports/macports-ports/blob/master/emulators/wine-devel/... Locally I have the updated version of MR 5871 Not sure, but maybe the `envp` passed to `__wine_main` is NULL for some reason. I'm not sure how this would happen though (it happened in the past when I screwed up the preloader, but now the preloader isn't built at all).
`/opt/local/bin/wine-preloader` is not present, right? Also you must have done this somehow, but you ran `autoreconf` to regenerate `configure` and `include/config.h.in`? -- https://gitlab.winehq.org/wine/wine/-/merge_requests/5884#note_73859
`/opt/local/bin/wine-preloader` is not present, right?
Correct
Also you must have done this somehow, but you ran `autoreconf` to regenerate `configure` and `include/config.h.in`?
Yep I ran `autoreconf -i` so both `configure` and `include/config.h.in` were both updated correctly, even downgraded my copy of `autoconf` to match the one used by the Winehq actions. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/5884#note_73860
On Thu Jun 20 21:50:09 2024 +0000, Dean M Greer wrote:
`/opt/local/bin/wine-preloader` is not present, right? Correct Also you must have done this somehow, but you ran `autoreconf` to regenerate `configure` and `include/config.h.in`? Yep I ran `autoreconf -i` so both `configure` and `include/config.h.in` were both updated correctly, even downgraded my copy of `autoconf` to match the one used by the Winehq actions. I installed MacPorts, added a local repo, and copied the wine-devel Portfile+patches to it. I did remove `/Library/Developer/CommandLineTools` and reinstalled the latest version, but `xcode-select -p` is still set to Xcode 15.4. I did had to remove the vulkan and MoltenVK references from the Portfile in order to build (I ended up with an ARM64-only MoltenVK causing the build to fail). After that it installed and worked correctly.
I then added the patches from this MR (plus the `configure` and `config.h.in` update) ([nopreloader_mp.patch](/uploads/2aeadad496aaa3f73f3d4c701ad201c6/nopreloader_mp.patch)), reinstalled, confirmed it's correct (no preloader) and it's working for me. Could you try building with that patch? -- https://gitlab.winehq.org/wine/wine/-/merge_requests/5884#note_73873
On Fri Jun 21 05:24:38 2024 +0000, Brendan Shanks wrote:
I installed MacPorts, added a local repo, and copied the wine-devel Portfile+patches to it. I did remove `/Library/Developer/CommandLineTools` and reinstalled the latest version, but `xcode-select -p` is still set to Xcode 15.4. I did had to remove the vulkan and MoltenVK references from the Portfile in order to build (I ended up with an ARM64-only MoltenVK causing the build to fail). After that it installed and worked correctly. I then added the patches from this MR (plus the `configure` and `config.h.in` update) ([nopreloader_mp.patch](/uploads/2aeadad496aaa3f73f3d4c701ad201c6/nopreloader_mp.patch)), reinstalled, confirmed it's correct (no preloader) and it's working for me. Could you try building with that patch? Cleared ccache and rebuilt now it's working.
However it seems there's additional requirements to cherry pick this to wine-stable-9.0.x -- https://gitlab.winehq.org/wine/wine/-/merge_requests/5884#note_73926
On Fri Jun 21 14:46:03 2024 +0000, Dean M Greer wrote:
Cleared ccache and rebuilt now it's working. However it seems there's additional requirements to cherry pick this to wine-stable-9.0.x As for the issue with MacPorts I'll look into that as it should install MoltenVK as +universal in your case.
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/5884#note_73928
participants (4)
-
Brendan Shanks -
Brendan Shanks (＠bshanks)
-
Dean M Greer (＠Gcenx)
-
Marvin