https://bugs.winehq.org/show_bug.cgi?id=52761
Bug ID: 52761 Summary: mscoree:mscoree test fails without a display or console Product: Wine Version: 7.5 Hardware: x86-64 OS: Linux Status: NEW Severity: normal Priority: P2 Component: -unknown Assignee: wine-bugs@winehq.org Reporter: madewokherd@gmail.com Distribution: ---
The mscoree:mscoree test fails if you run it like this: $ DISPLAY= wine mscoree_test.exe mscoree </dev/null >log 2>&1
Without the DISPLAY=, it makes a bunch of console windows. That doesn't seem to happen on Windows, but that could be caused by our csc.exe.
https://bugs.winehq.org/show_bug.cgi?id=52761
Esme Povirk madewokherd@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Keywords| |download, regression, | |source, testcase
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #1 from Esme Povirk madewokherd@gmail.com --- This breaks github CI, and it indicates console windows are being created when they shouldn't be, so I consider this a blocker for wine-mono release at least until it's understood enough to know we don't have to address it there.
https://bugs.winehq.org/show_bug.cgi?id=52761
Esme Povirk madewokherd@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |madewokherd@gmail.com
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #2 from Esme Povirk madewokherd@gmail.com --- f034084d49b354811096524d472ae5172ac1cebf is the first bad commit commit f034084d49b354811096524d472ae5172ac1cebf Author: Eric Pouech eric.pouech@gmail.com Date: Thu Mar 17 08:27:28 2022 +0100
kernelbase: Handle corner case in CreateProcess.
In CreateProcess, if: - parent isn't attached to a console - CreateProcess's flag isn't set with DETACHED_PROCESS nor CREATE_NEW_CONSOLE - child is a CUI program then a console must be allocated for the child.
Wine-Bug: https://bugs.winehq.org/show_bug.cgi?id=52048 Signed-off-by: Eric Pouech eric.pouech@gmail.com Signed-off-by: Jacek Caban jacek@codeweavers.com Signed-off-by: Alexandre Julliard julliard@winehq.org
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #3 from Esme Povirk madewokherd@gmail.com --- I'm not sure which CreateProcess causes the console to be created. It could be this:
https://source.winehq.org/git/wine.git/blob/HEAD:/dlls/mscoree/tests/mscoree...
or this:
https://github.com/madewokherd/wine-mono/blob/develop/tools/csc-wrapper/csc-...
Since Windows doesn't seem to create these extra console windows, and the mscoree test code is the same, I have to assume it's the CreateProcess in csc-wrapper that creates a console window.
I didn't originally write csc-wrapper.cs, that was from Fabian Maurer, so I don't know why we use these settings in particular to create the process. AFAICT the "OutputDataReceived" handler doesn't actually work.
What we ideally want is for the child process to inherit the current process's stdout and not create its own console (unless csc-wrapper would create one, in which case it should use the same one). Maybe that means we should AttachConsole before creating the process?
(Though really, ideally we should have a csc.exe that works without creating another process.)
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #4 from Esme Povirk madewokherd@gmail.com --- csc-wrapper has CUI subsystem though, so it should automatically get a console, right?
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #5 from Esme Povirk madewokherd@gmail.com --- I guess on Windows, if we can get output from mscoree_test.exe, it has a console, so csc-wrapper and mcs inherit that. Is there any way we can create an analogous situation, where a CUI exe is created without a console, on Windows?
https://bugs.winehq.org/show_bug.cgi?id=52761
Esme Povirk madewokherd@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |eric.pouech@orange.fr, | |jacek@codeweavers.com
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #6 from Eric Pouech eric.pouech@orange.fr --- Created attachment 72130 --> https://bugs.winehq.org/attachment.cgi?id=72130 patch
does the attached patch solve the issue? (tested locally it no longer creates the additional consoles; but since I don't have .NET installed, the tests fail on missing .NET; so cannot tell if that's ok)
https://bugs.winehq.org/show_bug.cgi?id=52761
Eric Pouech eric.pouech@orange.fr changed:
What |Removed |Added ---------------------------------------------------------------------------- Attachment #72130|0 |1 is obsolete| |
--- Comment #7 from Eric Pouech eric.pouech@orange.fr --- Created attachment 72131 --> https://bugs.winehq.org/attachment.cgi?id=72131 patch
(simplified version of 72130:patch)
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #8 from Jacek Caban jacek@codeweavers.com --- The core of the problem is that we're unable to create a window for the console. For CI, it may be good to simply use null display driver, something like this should do:
wine reg add 'HKCU\Software\Wine\Drivers' /v Graphics /t REG_SZ /d null /f
We don't use it as an implicit fallback under the assumption that in most case it user error that we want to make verbose. We could achieve a similar effect just for consoles by ignoring CreateWindow() failure in conhost.
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #9 from Esme Povirk madewokherd@gmail.com --- The tests don't seem to create a console on Windows, so they shouldn't need to on Wine either. This suggests there's likely a regression where other things will create console windows that shouldn't.
If I just wanted a work-around for the CI, I could give it an X11 display, but it suggests to me that something is actually broken and therefore the CI should fail in this case.
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #10 from Esme Povirk madewokherd@gmail.com --- (In reply to Eric Pouech from comment #7)
Created attachment 72131 [details] patch
(simplified version of 72130:patch)
Still fails with the patch.
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #11 from Esme Povirk madewokherd@gmail.com --- Is there any way I can start mscoree_test.exe on Windows that's analogous to starting it this way (with no console and redirected std*) in Wine? If so, I should test that on Windows to see if it creates cmd windows (which would mean this is correct behavior and I should fix the CI). If not, then it's a broken state and Wine shouldn't do that or at the very least we should never start Wine that way.
https://bugs.winehq.org/show_bug.cgi?id=52761
Esme Povirk madewokherd@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Regression SHA1| |f034084d49b354811096524d472 | |ae5172ac1cebf
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #12 from Eric Pouech eric.pouech@orange.fr --- (In reply to Esme Povirk from comment #11)
Is there any way I can start mscoree_test.exe on Windows that's analogous to starting it this way (with no console and redirected std*) in Wine? If so, I should test that on Windows to see if it creates cmd windows (which would mean this is correct behavior and I should fix the CI). If not, then it's a broken state and Wine shouldn't do that or at the very least we should never start Wine that way.
CreateProcess with DETACHED_PROCESS flag the difference between wine and windows: - under windows, when launching mscoree_test.exe, as it belongs to the CUI subsystem, Windows create a console and attaches the process to it. this console is then interited by all child process (if any) - under wine, when launching mscore_test.exe + if wine detects a tty (A), Wine creates a windows's console attached to the unix tty (== where the shell inputs / outputs), and we're in a similar situation as on windows + otherwise, Wine doesn't create any console. in your case, subsequent creation of child process (since there's no console) will generate the creation of new console (B) (which fails as DISPLAY is unset)
(A) the detection is made whether at least one of input stream (fd=0), output stream (fd=1) or error stream (fd=2) points to the tty (where the shell inputs/outputs) in #1, as 0=/dev/null, 1,2=log => the detection fails (B) there are several kind of consoles on wine to make things nicer. Either it transparently inputs/outputs to a unix tty; or it creates a windows (user32.dll). The first kind should work (even when DISPLAY is unset). Second requires a graphic driver (either X11 when DISPLAY is set, or as Jacek #8 to use a null graphic driver)
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #13 from Eric Pouech eric.pouech@orange.fr --- (In reply to Esme Povirk from comment #10)
(In reply to Eric Pouech from comment #7)
Created attachment 72131 [details] patch
(simplified version of 72130:patch)
Still fails with the patch.
does it fail in the same way as in #1 (ie new consoles created?)
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #14 from Esme Povirk madewokherd@gmail.com --- I get nodrv_CreateWindow errors followed by a failure to write output, so it seems like it.
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #15 from Esme Povirk madewokherd@gmail.com --- OK, more generally, suppose I want to write a Linux script that calls a Windows command-line utility, controlling all std handles. If I understand correctly, the Windows command-line utility's output will go to the std handles, but any CUI child processes will create a console window instead of using my captured std handles. If I want to capture the std streams of child processes as well (as would happen on Windows if I started the executable from cmd.exe with capture, or from WSL), what is the correct way to invoke Wine to accomplish this?
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #16 from Jacek Caban jacek@codeweavers.com --- STARTF_USESTDHANDLES gives you full control over std handles and when specified, it overrides default handling. When it's not specified, default handling depends on other factors and Windows either inherits handles from parent process or fills them with console handles (or no handles at all). If you want to create a process with no console, but redirected std handles, then a mix of STARTF_USESTDHANDLES and DETACHED_PROCESS should do the trick. This article describes it in great details (and was mostly accurate for cases that I tested): https://github.com/rprichard/win32-console-docs/blob/master/README.md
The problem I described in comment 8 is more general than this test and the problem may affect other CI-alike scenarios as well. I also noticed that mscoree tests additionally use CREATE_NO_WINDOW flag, which is not implemented on Wine. My understanding (I didn't test it) is that this flag should cause console to be created without a visible window (so it's essentially an invisible instance of conhost.exe process). That's another problem that would ideally be fixed as well.
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #17 from Esme Povirk madewokherd@gmail.com --- Created attachment 72141 --> https://bugs.winehq.org/attachment.cgi?id=72141 test exe launcher
Thanks, I've made a test program to recreate this in Windows. For some reason, it doesn't seem to capture stdout/stderr at all, but it's enough to check how this behaves.
When running mscoree_test.exe through this executable, it creates console windows, but most are for loadpaths.exe so I'm not sure if we get them for csc.exe or not.
https://bugs.winehq.org/show_bug.cgi?id=52761
Esme Povirk madewokherd@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Component|-unknown |mscoree
--- Comment #18 from Esme Povirk madewokherd@gmail.com --- With the test modified to prevent the loadpaths.exe windows, Windows does create a window for csc.exe processes. So the mscoree tests are broken.
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #19 from Esme Povirk madewokherd@gmail.com --- Setting DETACHED_PROCESS in compile_cs prevents console windows from being created for csc.exe on Windows and Wine, but in Wine we get a console window for mcs.exe, so there's more that needs fixed. It probably requires a change to how we call mcs.exe.
Ideally I'd like to get rid of this "RedirectStandardOutput" thing, is there any way we can call mcs.exe so it'll inherit handles and not create a new console window?
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #20 from Esme Povirk madewokherd@gmail.com --- Redirection in .NET doesn't seem to be sufficient to prevent the creation of a console.
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #21 from Esme Povirk madewokherd@gmail.com --- I sent a few patches that prevent the test from creating console windows on Windows, but not fully on Wine as we still get them for mcs.exe processes.
I also pushed a change to Wine Mono to handle redirection in a more sound way: https://github.com/madewokherd/wine-mono/commit/c0bd3ce453d55a17bb7a69135d9e...
That seems to fix the test failure, but it doesn't prevent the mcs.exe windows from popping up. In theory, CREATE_NO_WINDOW should fix that, but the attached patch doesn't seem to help.
https://bugs.winehq.org/show_bug.cgi?id=52761
François Gouget fgouget@codeweavers.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |fgouget@codeweavers.com
--- Comment #22 from François Gouget fgouget@codeweavers.com --- See also bug 52771.
https://bugs.winehq.org/show_bug.cgi?id=52761
Eric Pouech eric.pouech@orange.fr changed:
What |Removed |Added ---------------------------------------------------------------------------- Attachment #72131|0 |1 is obsolete| |
--- Comment #23 from Eric Pouech eric.pouech@orange.fr --- Created attachment 72147 --> https://bugs.winehq.org/attachment.cgi?id=72147 new version of patch
https://bugs.winehq.org/show_bug.cgi?id=52761
Eric Pouech eric.pouech@orange.fr changed:
What |Removed |Added ---------------------------------------------------------------------------- Attachment #72147|0 |1 is obsolete| |
--- Comment #24 from Eric Pouech eric.pouech@orange.fr --- Created attachment 72148 --> https://bugs.winehq.org/attachment.cgi?id=72148 new version of patch
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #25 from Eric Pouech eric.pouech@orange.fr --- - with the attached patch, I'm able locally to run the mscoree tests with ./wine mscoree_test.exe mscoree -o log2 </dev/null >& log, and without additional consoles being created. the log2 file show no 'test' errors, even if log contains lots of
Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'libloadpaths, Version=0.0.0.0, Culture=en, PublicKeyToken=null' or one of its dependencies.^M File name: 'libloadpaths, Version=0.0.0.0, Culture=en, PublicKeyToken=null' [ERROR] FATAL UNHANDLED EXCEPTION: System.IO.FileNotFoundException: Could not load file or assembly 'libloadpaths, Version=0.0.0.0, Culture=en, PublicKeyToken=null' or one of its dependencies.^M
(not sure if it's expected or not)
- unsetting DISPLAY will not work as some other processes launched during startup require creating a window (and it's not conhost)
- note: the patch tackles two issues: + because you redirect also stdin to /dev/null... + because in here: https://github.com/madewokherd/wine-mono/blob/develop/tools/csc-wrapper/csc-... the use of 'process.StartInfo.CreateNoWindow = true;' asks for the creation of the console which is not seen, and which may require your stream reader stuff to copy again the information...
so could you, separately, - test in csc wrapper, what gives the suppression of CreateNoWindow flag (it may be you could get rid of the stream copy bits) - test if the patch I attached helps in your integration (and without the patches you sent yesterday, they shouldn't be needed IMO)
in reply to #19: running wine ... >& log < /dev/null (or without the < /dev/null part) should redirect all output of ... and all the subprocesses into log, unless for the subprocesses created with DETACHED_PROCESS, CREATE_NO_WINDOW or CREATE_NEW_WINDOW flag... so in theory, with the attached patch + removal of CreateNoWindow in csc wrapper, you should be able to capture all output... (famous last words <g>)
https://bugs.winehq.org/show_bug.cgi?id=52761
Austin English austinenglish@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |austinenglish@gmail.com
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #26 from Esme Povirk madewokherd@gmail.com --- The patches I sent yesterday are required to prevent the creation of additional console windows *on Windows* if mscoree_test.exe is created with DETACHED_PROCESS.
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #27 from Eric Pouech eric.pouech@orange.fr --- the point being why do you need to pass DETACHED_PROCESS at all?
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #28 from Esme Povirk madewokherd@gmail.com --- To simulate what's happening on Wine. Keep in mind that a Windows program could also do this, and it shouldn't make an extra console window when it starts csc.exe.
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #29 from Eric Pouech eric.pouech@orange.fr --- it will when the first process is not connected to a console object. that's why I'd recommand: - do not use DETACH_PROCESS, no CREATE_NO_WINDOW flags. it will make you life harder - on wine, use the proposed patch, and run with ./wine foo.exe >& log < /dev/null (the redirection of input being potentially optional, depend on what you need on input) - on windows, depending on how your process is started (not clear how the CI tool does it), you can wrap it inside a cmd.exe to ensure console creation
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #30 from Esme Povirk madewokherd@gmail.com --- I can change the Wine tests, but I can't control how Windows programs invoke csc.exe. I think it's not only possible but very likely that some Windows programs start csc.exe with DETACHED_PROCESS. Currently, on Wine, that results in a new console window, not for csc.exe but for mcs.exe. So I need a way I can invoke mcs.exe that won't create a new console window regardless of how csc.exe was started, and also not lose mcs.exe's output in the process.
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #31 from Eric Pouech eric.pouech@orange.fr --- (I'm not proficient in c#/.net; I assume csc if the MS c# compiler, while mcs is mono's?) so you're comparing: - windows: csc.exe invocation => no new console window - wine: mcs.exe invocation => new console window (mcs being used as a replacement of csc)
is the invocation done only from here https://github.com/madewokherd/wine-mono/blob/develop/tools/csc-wrapper/csc-... or from other locations?
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #32 from Esme Povirk madewokherd@gmail.com --- Any program can call c:\windows\microsoft.net...\csc.exe. csc-wrapper.cs gets compiled and installed there as a replacement. So programs will call csc-wrapper directly as csc.exe, the way the mscoree test does, and csc-wrapper works by calling mcs.exe.
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #33 from Eric Pouech eric.pouech@orange.fr --- afaics in https://github.com/madewokherd/wine-mono/blob/develop/tools/csc-wrapper/csc-..., what happens is: caller (A) - csc_wrapper (new process) (B) - csc (or mcs)
the (B) call to CreateProcess uses the CREATE_NO_WINDOW flag which is not implemented in Wine
so if caller (A) creates csc_wrapper without a console (DETACHED_PROCESS), this will trigger the generation of a console when creating mcs process
normally, with the patchset sent earlier on to wine devel, support for CREATE_NO_WINDOW should be implemented, and this would work without changing what you've already changed
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #34 from Esme Povirk madewokherd@gmail.com --- Maybe I should just wait for the console code to stabilize and retest then?
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #35 from Eric Pouech eric.pouech@orange.fr --- yes and from the two patches you sent, the one which adds the DETACHED_PROCESS flag is the one to be reconsidered IMO AFAICT, the loadpaths.exe is just created in tests to test registering some .net bits. no output to console or std stream from this one is used, so that's fine
OTOH, csc.exe should emit warnings/error reports, at least to std output, and potentially console when present. so running it detached is a bad option IMO, and we'd better ensure its run attached to the console
https://bugs.winehq.org/show_bug.cgi?id=52761
Julian Rüger jr98@gmx.net changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |jr98@gmx.net
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #36 from François Gouget fgouget@codeweavers.com --- I can run the command below without getting any failure:
DISPLAY= ./wine ./dlls/mscoree/tests/mscoree_test.exe mscoree </dev/null >log 2>&1
Does that mean that eff83cd4e00e fixed this issue? Should this bug be closed?
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #37 from Esme Povirk madewokherd@gmail.com --- Just started a CI run to make sure. https://github.com/madewokherd/wine-mono/actions/runs/2283162642
https://bugs.winehq.org/show_bug.cgi?id=52761
Esme Povirk madewokherd@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |FIXED
--- Comment #38 from Esme Povirk madewokherd@gmail.com --- Yep, seems to be failing only for other reasons now.
https://bugs.winehq.org/show_bug.cgi?id=52761
--- Comment #39 from Esme Povirk madewokherd@gmail.com --- (In reply to Esme Povirk from comment #30)
So I need a way I can invoke mcs.exe that won't create a new console window regardless of how csc.exe was started, and also not lose mcs.exe's output in the process.
In case anyone in a similar situation finds this bug and, the answer appears to be:
Check whether the process has a console attached (GetConsoleCP will return non-zero if it does). If no: Use the CREATE_NO_WINDOW flag with CreateProcess and STARTF_USESTDHANDLES in STARTUPINFO. Set hStd* in STARTUPINFO to the corresponding handle from GetStdHandle. If yes: Call CreateProcess without DETACHED_PROCESS, CREATE_NEW_CONSOLE, CREATE_NO_WINDOW, or STARTF_USESTDHANDLES.
This *might* work with DETACHED_PROCESS instead of CREATE_NO_WINDOW, since in this case mcs.exe doesn't have to create any child processes. If there's a possibility of further descendant console processes, and the immediate child window doesn't call CreateProcess in the same way, those further descendants would create a new console window when DETACHED_PROCESS is used.
https://bugs.winehq.org/show_bug.cgi?id=52761
Alexandre Julliard julliard@winehq.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |CLOSED
--- Comment #40 from Alexandre Julliard julliard@winehq.org --- Closing bugs fixed in 7.9.