Hiya,
Firstly, fantastic work, and its also explained to me something Eric said which I didn't grasp...
While preparing tests I found the 'user' backend of wineconsole works with WriteConsole/WriteFile correct (see test1 and test2), so I used it as etal. I figured out the main issue with cmd: It passes strings to WriteFile in ANSI, but should in OEM. I think the main locale issue is found. CMD and XCOPY violate this. So it is not MSVCRT bug. Moreover, [w]fprintf MUST NOT perform any
AnsiToOem
convertions (as test3 and test4 show).
I made and attached a quick hack to demostrate that cmd was buggy. Attached
screenshots show the difference. The patch heals almost everything, even localized filenames. So I'm CC'ing mail to wine-devel, console/cmd gurus must know much more ;-)
I found some issues with wprintf, see README for test3 and test4. The attached tests are made with your patch applied.
Another interesting url is here, which confirms what you are saying http://smallcode.weblogs.us/2006/10/25/code-page-for-win32-console-programs/
So, another round of discussion but things have moved forward a lot...
1. The underlying problem appears to be that the output from both programs (and any other command line program) should not be done through msvcrt's i/o functions or writeconsoleA/writefileA, it needs to be either converted to OEM and then printf/writefile/writeconsole'd.
Ideally, a Unicode string would work better, which should be writeconsolew'd / writefile'd if that fails (eg if output is redirected to a file). => The only problem with this I can see is that this would result in redirected output containing Unicode which is wrong. However, advice I found on a URL on the web said this:
>>>>>>>>>>
Tips and considerations:
- use WriteConsole to output Unicode strings. Note that this API works
only on console handles and can not be used for a redirection to a disk file.
- If the output is being redirected to a disk file, use WriteFile with
the current console code page that can be retrieved by GetConsoleOutputCP (the console code page might be different from the currently selected OEM code page!).
>>>>>>>>>>
So I believe the output function in cmd.exe should end up (When fully unicoded):
writeconsoleW if this fails Convert from wide to multibyte using consoleoutputpt Writefile the result endif
Temporarily, since in cmd.exe we have an ANSI string in our hands, use something like the mechanism you have coded using chartooem. Out of interest since the string is in ansi, the msdn says CharToOEM(A) can convert in place, so if you put CharToOem(message, message); just before the WriteFiles (and remove the const qualifier), does this work?
2. The testcases I have previously confirm that msvcrt's functions are also misfunctioning, and I strongly believe my current solution to those is correct. Ie for applications using msvcrt wprintf functionality it needs to take into account the mode the file was opened. (I don't know which other msvcrt routines have similar impacts, but if this is accepted I'll try to take a look) => Unless I get any negative comments soon I will tidy the tests up and submit that as a patch
3. The right solution is that cmd.exe works in Unicode, which is an exercise I plan to do as soon as I have finished work on the few remaining issues I want to address (I want to look at attrib, for and copy, plus a few bugs I have written on scraps of paper...)
4. xcopy needs a similar fix - If you are happy to do some tests (especially xcopying files with russian names, plus copying directories created with Russian names) I'll contact you directly with a patch to test for me
5. Once all the above is done, I'd like to check on your test3/4 cases to see if there's any residual problems.
Again, thanks for your excellent work. I never thought I'd be so pleased to see Russian characters on a screen...!
Jason