[Bug 60112] New: msvcrt: fopen incorrectly uses CreateFileW instead of CreateFileA

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=60112

            Bug ID: 60112
           Summary: msvcrt: fopen incorrectly uses CreateFileW instead of
                    CreateFileA
           Product: Wine
           Version: 11.14
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: msvcrt
          Assignee: [e-mail]
          Reporter: [e-mail]
  Target Milestone: ---
      Distribution: ---

Created attachment 81671
  --> http://bugs.winehq.org/attachment.cgi?id=81671
Standalone MSVCRT narrow fopen reproducer

This bug affects some FitGirl repack installers that rely on intercepting
`CreateFileA` while reading their installation data.

When a program calls `msvcrt.fopen()` with a narrow path, Wine converts the
path to UTF-16 and opens it through `CreateFileW`. Windows MSVCRT keeps the
narrow call on `CreateFileA`. Programs that replace the imported `CreateFileA`
entry therefore work on Windows but fail on Wine.

The standalone reproducer hooks the `CreateFileA` and `CreateFileW` imports in
`msvcrt.dll`. Its `CreateFileA` hook maps the synthetic path
`wine_fopen_virtual_input` to an anonymous pipe containing one byte. No third
party DLL or application data is used.

Unpatched Wine reports:

    fopen_route_summary | hookA=1 hookW=1 callsA=0 callsW=1 opened=0 read=0
byte=0 errno=2 win32Error=2 pass=0

Windows reports:

    fopen_route_summary | hookA=1 hookW=1 callsA=1 callsW=0 opened=1 read=1
byte=81 errno=0 win32Error=0 pass=1

Wine with the candidate patch reports:

    fopen_route_summary | hookA=1 hookW=1 callsA=1 callsW=0 opened=1 read=1
byte=81 errno=0 win32Error=0 pass=1

The focused unpatched trace shows the narrow `fopen()` call reaching the wide
path and failing as a literal file lookup:

    trace:msvcrt:_wfsopen (L"wine_fopen_virtual_input",L"rb")
    trace:file:CreateFileW L"wine_fopen_virtual_input" GENERIC_READ
FILE_SHARE_READ FILE_SHARE_WRITE creation 3 attributes 0x1
    warn:file:CreateFileW Unable to create file L"wine_fopen_virtual_input"
(status c0000034)

The first incompatible behavior is in `dlls/msvcrt/file.c`: `_fsopen()`
converts its narrow path with `wstrdupa_utf8()` and delegates to `_wfsopen()`,
which reaches `_wsopen()` and `CreateFileW`.

This bypasses the program supplied `CreateFileA` hook. The synthetic path is
then treated as a literal file name, `fopen()` returns NULL, and the virtual
input cannot be read.

Reproduce:

1. Build `msvcrt_fopen_route_probe.c` as a 64-bit console executable with a
MinGW compiler.
2. Run the executable on Windows and observe `callsA=1`, `callsW=0`, and
`pass=1`.
3. Run the same executable with unpatched Wine and observe `callsA=0`,
`callsW=1`, and `pass=0`.

The candidate patch keeps narrow `_fsopen()` and `_sopen()` calls on
`CreateFileA` while leaving the wide entry points on `CreateFileW`. It also
adds an MSVCRT test that verifies which imported Win32 function receives a
narrow `fopen()` call.

Testing:

- The standalone executable passes on Windows 10.0.19045.3324.
- The standalone executable fails on Wine 11.14.
- The standalone executable passes on Wine 11.14-14-g84fe968b936 with the
candidate patch.
- The 32-bit and 64-bit MSVCRT file test fails at the new assertions without
the implementation change: `CreateFileA called 0 times` and `CreateFileW called
1 times`.
- The 32-bit and 64-bit MSVCRT file test passes when the candidate patch is
applied.

I have attached a tar archive containing the reproducer source code and
prebuilt binary, the native Windows result, Wine logs from before and after the
fix, the focused debug trace, the candidate patch, and the build and run
scripts.

-- 
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.