Starting with macOS 10.15, `mmap()` for a file with `PROT_EXEC` fails with `EPERM`. But for some reason, doing separate `mmap()` and then `mprotect()` calls works.
This fixes `NtUserRegisterClassExWOW: Failed to get shared session object for window class` errors seen when running a 32-bit EXE lacking the NX compat bit. The shared memory object being used for window classes was being mapped executable, this was failing because of the above, and then `map_file_into_view()` was falling back to `pread()` which doesn't make sense for a shared memory region. (It seems like a bug to use `pread()` in this scenario rather than return an error, although I'm not sure how that could be detected).
I don't love the preprocessor black-magic to replace every mmap() call, but it avoids having to modify any other functions. If we want to avoid that, `map_pe_header()` and `map_file_into_view()` are the `mmap()` calls that I know need to be changed.
CrossOver has used an almost-identical hack for years.
-- v2: ntdll: On macOS, use separate mmap() and mprotect() calls when mapping files with PROT_EXEC.