[Bug 60344] New: msvcrt: stdio functions crash on a NULL stream instead of failing with EINVAL (VCV Rack 2, Cardinal)

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=60344

            Bug ID: 60344
           Summary: msvcrt: stdio functions crash on a NULL stream instead
                    of failing with EINVAL (VCV Rack 2, Cardinal)
           Product: Wine
           Version: 11.17
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: msvcrt
          Assignee: [e-mail]
          Reporter: [e-mail]
  Target Milestone: ---
      Distribution: ---

fread() and most other stdio functions in dlls/msvcrt/file.c lock the stream
before validating it. With a NULL stream this ends in
RtlEnterCriticalSection(NULL + 0x30) and a page fault on read access to 0x50.
fclose(), ungetc(), ungetwc() and setvbuf() already use
MSVCRT_CHECK_PMT(file != NULL), the others do not.

Native msvcrt.dll (Windows 11, build 26200) returns an error and sets errno to
EINVAL for all of them: fread/fwrite 0, fgets/fgetws NULL,
fgetc/fputc/fputs/_getw/_putw EOF, fgetwc/fputwc/fputws WEOF,
ftell/fseek/_fileno -1, feof/ferror 0, and rewind/clearerr just set errno.
_fseeki64 is the one exception, it crashes on Windows too. ucrtbase terminates
the process by default (0xC0000409) and behaves like msvcrt once an invalid
parameter handler is set. I measured this with a small freestanding program,
one process per function. Under Wine, 21 of those 25 functions crash with
0xC0000005. Tested with wine-11.17 (unmodified distribution package, fresh
prefix) and with a wine-11.13 based build.

Real-world impact: VCV Rack 2 (Free and Pro, Windows builds, mingw, linked
against msvcrt.dll) and its fork Cardinal crash whenever they archive a patch
directory. rack::system::archiveDirectory() calls fopen() on every libarchive
entry, including directories, gets NULL for those and passes it to fread():
https://github.com/VCVRack/Rack/blob/v2/src/system.cpp#L477
That is a bug in Rack, but it is harmless on Windows. Under Wine, Rack
standalone dies on "Save", and as a VST3/VST2/CLAP plugin it takes the host
down as soon as it is instantiated (Ableton Live 12: "A serious program error
has occurred"). Reports that were never root-caused:
https://github.com/DISTRHO/Cardinal/issues/275
https://github.com/DISTRHO/Cardinal/issues/854
https://community.vcvrack.com/t/vcv-rack-2-free-closes-when-i-save-any-patch...

Steps to reproduce without any third-party software: LoadLibrary("msvcrt.dll"),
GetProcAddress("fread"), call fread(buf, 1, 16, NULL).
Result: unhandled page fault on read access to 0x50 in ntdll
(RtlEnterCriticalSection+7). With Rack Free 2.6.6 (public download): run
Rack.exe and use File > Save as.

No patch attached: I did this analysis with an AI coding assistant, and Wine
does not accept LLM-generated code. The fix looks like the same
MSVCRT_CHECK_PMT(file != NULL) check that fclose() has. For what it is worth,
such a change was built and tested here: a test with the values above passes
on Windows 11 against native msvcrt.dll, crashes on unpatched Wine 11.13 and
passes with the checks in place (msvcrt file test, 1147 tests, 0 failures),
and an unmodified libRack.dll 2.6.6 then archives correctly. Not measured:
msvcr80 to msvcr120.

Measurements and reproducers (build with clang and lld alone):
https://github.com/slein87/vcv-rack-wine-fix
Please do not copy the diff in its wine/ directory, it is AI-generated and
only documents what was tested.

-- 
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.