[Bug 55200] New: MSVC cl.exe misbehaving (buggy/inconsistent behaviour) wrt include paths since "ntdll: Default to Windows 10"

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=55200

Bug ID: 55200
           Summary: MSVC cl.exe misbehaving (buggy/inconsistent behaviour)
                    wrt include paths since "ntdll: Default to Windows 10"
           Product: Wine
           Version: 8.1
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: -unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
   Regression SHA1: 69154f0329aec4fb64886a0689da198b5323dcde
      Distribution: ---

Background: I package MSVC with Wine wrapping, so that people can compile
software using MSVC on Unix systems - see
https://github.com/mstorsjo/msvc-wine.

Since Wine 8.1, specifically commit 69154f0329aec4fb64886a0689da198b5323dcde
"ntdll: Default to Windows 10", cl.exe in Wine started misbehaving with respect
to how it finds headers in some include paths. The misbehaviour isn't entirely
consistent and only appears in some fairly specific cases. (When compiling a
project of the size of LLVM, with ~3000 compiled translation units, the bug
appears in 3 of these translation units.)

I've managed to reproduce the situation with a fairly minimal testcase. To
reproduce, first install a fresh version of msvc-wine:

$ git clone https://github.com/mstorsjo/msvc-wine
$ cd msvc-wine
$ ./vsdownload.py --dest $HOME/msvc2022-17.6 --accept-license
$ ./install.sh $HOME/msvc2022-17.6

Then set up a minimal mock source project structure:

$ mkdir -p msvc-include-bug/subdir
$ touch msvc-include-bug/subdir/header.h
$ echo '#include "subdir/header.h"' > msvc-include-bug/subdir/file.cpp

Now test preprocessing file.cpp to see that it manages to find the header.
Bypass the msvc-wine wrapper scripts and call it with wine manually:

$ . $HOME/msvc2022-17.6/bin/x64/msvcenv.sh # Set up the
INCLUDE/LIB/WINEDLLOVERRIDES env vars as necessary
$ WINEDEBUG=-all wine64
$HOME/msvc2022-17.6/vc/tools/msvc/14.36.32532/bin/Hostx64/x64/cl.exe -P
-Fipreproc.cpp -I$(pwd)/msvc-include-bug/subdir -I$(pwd)/msvc-include-bug
z:$(pwd)/msvc-include-bug/subdir/file.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 19.36.32532 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

file.cpp

This runs successfully, as reference example. Now update file.cpp to include
<corecrt_wio.h> before "subdir/header.h":

$ echo '#include <corecrt_wio.h>' > msvc-include-bug/subdir/file.cpp
$ echo '#include "subdir/header.h"' >> msvc-include-bug/subdir/file.cpp

Rerun the same compilation command:

$ . $HOME/msvc2022-17.6/bin/x64/msvcenv.sh
$ WINEDEBUG=-all wine64
$HOME/msvc2022-17.6/vc/tools/msvc/14.36.32532/bin/Hostx64/x64/cl.exe -P
-Fipreproc.cpp -I$(pwd)/msvc-include-bug/subdir -I$(pwd)/msvc-include-bug
z:$(pwd)/msvc-include-bug/subdir/file.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 19.36.32532 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

file.cpp
z:/home/martin/msvc-include-bug/subdir/file.cpp(2): fatal error C1083: Cannot
open include file: 'subdir/header.h': No such file or directory

For some unknown reason, the extra set of headers included by <corecrt_wio.h>
make cl.exe no longer find "subdir/header.h" even though
-I$(pwd)/msvc-include-bug is passed to the compiler. The bug only appears when
both $(pwd)/msvc-include-bug and $(pwd)/msvc-include-bug/subdir are added to
the include path.

$(pwd) expanding to an absolute unix path should be resolvable as an
drive-relative (implicitly relative to z:) within Wine.

Now if we clarify the include paths, that previously were set up as
-I$(pwd)/... into fully absolute windows paths as -Iz:$(pwd)/..., then the test
succeeds again:

$ WINEDEBUG=-all wine64
$HOME/msvc2022-17.6/vc/tools/msvc/14.36.32532/bin/Hostx64/x64/cl.exe -P
-Fipreproc.cpp -Iz:$(pwd)/msvc-include-bug/subdir -Iz:$(pwd)/msvc-include-bug
z:$(pwd)/msvc-include-bug/subdir/file.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 19.36.32532 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

file.cpp

Before the relevant Wine commit, this bug didn't appear at all in this case.

It's probably easy to work around this issue within msvc-wine by rewriting -I
arguments with absolute paths into Windows absolute paths like z:/... - but
it's curious why this is needed now when it wasn't needed before, and MSVC
cl.exe itself is acting very inconsistent here - it's only needed in this very
precise setup.

(For context, for future self: When compiling LLVM, this bug currently hits
three files under llvm/lib/Target/AArch64/MCTargetDesc -
AArch64WinCOFFObjectWriter.cpp, AArch64MCTargetDesc.cpp and
AArch64TargetStreamer.cpp.)