[Bug 9056] Mono 1.2.4 fails to find mscorlib.dll on startup

http://bugs.winehq.org/show_bug.cgi?id=9056

--- Comment #19 from Anastasius Focht   2007-09-25 09:38:40 ---
Hello,

thanks for the patches.
_wspawnvp() seems to unearth more mono bugs ;-(

--- snip ---
001b:Call msvcrt._wspawnvp(00000001,00155520
L"gspawn-win32-helper.exe",00155558) ret=673081ea
001b:Call KERNEL32.GetFileAttributesW(00155520 L"gspawn-win32-helper.exe")
ret=602fba80
001b:Ret  KERNEL32.GetFileAttributesW() retval=00000020 ret=602fba80
001b:Call KERNEL32.GetFullPathNameW(00155520
L"gspawn-win32-helper.exe",00000104,0061f5e4,00000000) ret=602fbaaa
001b:Ret  KERNEL32.GetFullPathNameW() retval=00000037 ret=602fbaaa
001b:Call KERNEL32.TlsGetValue(00000000) ret=603130a1
001b:Ret  KERNEL32.TlsGetValue() retval=0014b960 ret=603130a1
001b:Call KERNEL32.TlsGetValue(00000000) ret=603130a1
001b:Ret  KERNEL32.TlsGetValue() retval=0014b960 ret=603130a1
001b:Call ntdll.RtlAllocateHeap(00110000,00000000,00000194) ret=60305ab7
001b:Ret  ntdll.RtlAllocateHeap() retval=00155810 ret=60305ab7
001b:trace:msvcrt:_wspawnve :call (L"C:\Program
Files\Mono-1.2.5\bin\gspawn-win32-helper.exe"), params
(L"gspawn-win32-helper.exe 10 z 6 8 - y y w mono -O=all
--aot=outfile=c:windowsprofilesfocht.monoaot-cachemscorlib-FF1F1EDF-450A-4567-ADEC-30241E007373.dll
C:Program FilesMono-1.2.5libmono1.0mscorlib.dll"), env (Null)
001b:Call ntdll.RtlAllocateHeap(00110000,00000008,0000003b) ret=603059b7
001b:Ret  ntdll.RtlAllocateHeap() retval=001559b0 ret=603059b7
001b:Call KERNEL32.CreateProcessW(0061f350 L"C:\Program
Files\Mono-1.2.5\bin\gspawn-win32-helper.exe",00155810
L"gspawn-win32-helper.exe 10 z 6 8 - y y w mono -O=all
--aot=outfile=c:windowsprofilesfocht.monoaot-cachemscorlib-FF1F1EDF-450A-4567-ADEC-30241E007373.dll
C:Program
FilesMono-1.2.5libmono1.0mscorlib.dll",00000000,00000000,00000001,00000000,00000000,00000000,0061f558,0061f59c)
ret=6030ccdb
0023:trace:relay:init_debug_lists RelayExclude =
L"ntdll.RtlEnterCriticalSection;ntdll.RtlLeaveCriticalSection;kernel32.94;kernel32.95;kernel32.96;kernel32.97;kernel32.98"
0023:trace:relay:init_debug_lists RelayFromExclude =
L"winex11.drv;user32;gdi32;advapi32;kernel32"
0023:Call KERNEL32.__wine_kernel_init() ret=7bc43738
0023:Call PE DLL (proc=0x6031a360,module=0x602f0000
L"msvcrt.dll",reason=WINE_PREATTACH,res=(nil))
0023:trace:msvcrt:DllMain (0x602f0000, UNKNOWN, (nil)) pid(32), tid(23), tls(0)
0023:Ret  PE DLL (proc=0x6031a360,module=0x602f0000
L"msvcrt.dll",reason=WINE_PREATTACH,res=(nil)) retval=1
001b:Ret  KERNEL32.CreateProcessW() retval=00000001 ret=6030ccdb
...
001b:Ret  msvcrt._wspawnvp() retval=000000d0 ret=673081ea 
--- snip ---

I hex dumped argv[] to be sure there is no error in wine's
msvcrt_argvtos_wide(argv,' ') conversion.
It seems mono/glib messes up its command line when using windows style paths
(with spaces and backslashes).

I digged into their sources where they build command line and spawn AOT
compiler and found g_spawn_command_line_sync() call.
The glib documentation to this call:
http://library.gnome.org/devel/glib/unstable/glib-Spawning-Processes.html#g-...

--- quote ---
On Windows, please note the implications of g_shell_parse_argv() parsing
command_line. Parsing is done according to Unix shell rules, not Windows
command interpreter rules. Space is a separator, and backslashes are special.
Thus you cannot simply pass a command_line containing canonical Windows paths,
like "c:\program files\app\app.exe", as the backslashes will be eaten, and
the space will act as a separator. You need to enclose such paths with single
quotes, like "'c:\program files\app\app.exe' 'e:\folder\argument.txt'".
--- quote ---

This is not done in mono. The whole commandline args to argv[] parsing gets
messed up - with the result you see in wine trace.

Ok, there is something left on the wine side I mentioned before.
Could you please fix wine's FormatMessageA/W to check for NULL lpBuffer?

--- snip dlls/kernel32/format_msg.c ---
DWORD WINAPI FormatMessageA(
        DWORD   dwFlags,
        LPCVOID lpSource,
        DWORD   dwMessageId,
        DWORD   dwLanguageId,
        LPSTR   lpBuffer,
        DWORD   nSize,
        va_list* _args )
{ 
...
    if( !lpBuffer)
    {
        SetLastError( ERROR_INVALID_PARAMETER);
        return 0;
    }
...
}
--- snip dlls/kernel32/format_msg.c ---

Add the check to both, "A" and "W" Versions.
This prevents crashing with NULL lpBuffer argument supplied and matches windows
behaviour (windows returns invalid param).
Add a simple test case with FormatMessage() + lpBuffer = NULL to verify.

Regards