Unable to start UTF-8 encoded executable in UTF-8 locale - one line fix included

[Alex Villací­s Lasso]

I so wanted to be the first to provide the fix to the Open File dialog 
not handling UTF-8, but Michael Jung beat me at it :-/

While trying Michael Jung's fix, I found this other issue: when wine is 
run in an UTF-8 locale, and a native PE executable exists with an UTF-8 
encoded filename (for example gatón.exe with an [U+00F3 LATIN SMALL 
LETTER O WITH ACUTE]), any attempt to run this executable will fail with 
the following message:

wine: cannot open builtin library for <your executable name>: f

This suggests that wine somehow thinks the UTF-8 encoded filename is a 
builtin library instead of an actual PE file. So I dug into 
dlls/kernel/process.c and found the following:

* Function __wine_kernel_init() calls find_exe_file() with exe_nameW set 
to the Unicode version of the exename, and expects main_exe_file to be 
set for a native exe, or NULL for a builtin.
* Function find_exe_file() is supposed to try loading the file as 
builtin, then native. For the builtin try, it calls 
open_builtin_exe_file() to test whether a builtin exists with the 
specified name, and expects the flag file_exists to be set on return.
* Function open_builtin_exe_file() does a couple of sanity checks on the 
filename, including checking that all characters of the filename are in 
the 7-bit ASCII range (0..127), then delegates the file-exists test to 
wine_dll_load_main_exe().

The problem arises because open_builtin_exe_file() fails to set the 
file_exists flag either way when a sanity check fails. The Unicode 
version of the UTF-8 encoded exename fails the 7-bit ASCII test, but the 
function leaves file_exists uninitialized. Therefore the rest of the 
code thinks that the filename references a builtin (since file_exists is 
nonzero) and then the loading fails. The one-line fix attached simply 
sets file_exists to false *before* doing anything else in 
open_builtin_exe_file(), so that a filename that fails the sanity checks 
is properly flagged as non-existing.

Changelog:
* Initialize file_exists to 0 at exe load test, prevents mistaking of 
UTF-8 encoded exenames as builtins.

Alex Villacís Lasso

Unable to start UTF-8 encoded executable in UTF-8 locale - one line fix included

[Andreas Mohr]

Hi,

On Thu, Nov 24, 2005 at 12:09:06PM -0500, Alex Villací­s Lasso wrote:
> Changelog:
> * Initialize file_exists to 0 at exe load test, prevents mistaking 
> of UTF-8 encoded exenames as builtins.

Isn't that almost *exactly* what mengzhuo li very recently sent?
Is it the same place or the same problem in a different part of
process.c?
> I so wanted to be the first to provide the fix to the Open File
> dialog not handling UTF-8, but Michael Jung beat me at it :-/

Uhoh, so my mail is bad news here, I'm afraid ;-\

Andreas Mohr

[Alex Villací­s Lasso]

Andreas Mohr wrote:
> Hi,
>
> On Thu, Nov 24, 2005 at 12:09:06PM -0500, Alex Villací­s Lasso wrote:
> > Changelog:
> > * Initialize file_exists to 0 at exe load test, prevents mistaking 
> > of UTF-8 encoded exenames as builtins.
>
> Isn't that almost *exactly* what mengzhuo li very recently sent?
> Is it the same place or the same problem in a different part of
> process.c?

Now that I checked the patch by mengzhuo li, I realized that it fixes 
the exact same problem, but at a different place from my patch. My patch 
initializes the value at the callee, mengzhuo li's initializes it at the 
caller. Personally, I think that the proper place to fix it is at the 
callee, because it seems that the caller was initially written with the 
assumption that the callee unconditionally initializes the flag. Since 
the latest CVS did not include the patch, I assumed the bug was 
unacknowledged, and I did not think that the problem mengzhuo li saw was 
the same as what I saw. Especially because from his explanation, it 
seemed that it fixed an attempt to execute a non-executable file (with a 
.txt extension), rather than mistaking a PE native exe for a builtin 
(which is what I saw). These two problems are actually two symptoms of 
the same bug.

Alex Villacís Lasso