"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
In an attempt to get away from a possibly confusing subject line, and to converge on an agreed behavior, here is an attempt to clear the desired locale behavior Wine should have. Dmitry (and anyone else), please comment on this table where you think there are errors in it, and say what should be there.
In the "Unix setting", I'm always assuming that LC_ALL overrides whatever is written in that column, and that LANG is used if neither LC_ALL nor the column are set.
I said it many times already, and I'll repeat once more: Windows has only one active locale which defines all the things you mentioned in the table except language of the interface which let's set aside to not confuse each other. There is no need to try to find appropriate mappings for LC_TYPE or anything else you find in the output of the 'locale' command. But still (almost) any locale setting you want to modify in the win32 environment on a per user basis can be done either by running Control Panel/Regional Options, or modifying the registry or win.ini directly. By default all locale data are set according to the global system locale. Any attempt to have more than one active locale simultaneously will lead to confusion and nothing else, be it unix environment or win32. -- Dmitry.
Dmitry Timoshkov wrote:
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
In an attempt to get away from a possibly confusing subject line, and to converge on an agreed behavior, here is an attempt to clear the desired locale behavior Wine should have. Dmitry (and anyone else), please comment on this table where you think there are errors in it, and say what should be there.
In the "Unix setting", I'm always assuming that LC_ALL overrides whatever is written in that column, and that LANG is used if neither LC_ALL nor the column are set.
I said it many times already, and I'll repeat once more: Windows has only one active locale which defines all the things you mentioned in the table except language of the interface which let's set aside to not confuse each other. There is no need to try to find appropriate mappings for LC_TYPE or anything else you find in the output of the 'locale' command. But still (almost) any locale setting you want to modify in the win32 environment on a per user basis can be done either by running Control Panel/Regional Options, or modifying the registry or win.ini directly. By default all locale data are set according to the global system locale.
Ok. Then please do the following experiment for me, will ya? 1. Hover the mouse over the clock. The date is displayed. Watch the date display language. Now go to "Regional options", and change the setting to "English (U.S.)". Hover over the clock again - see how the date display setting changed? Obviously, the user locale affects the date format. 2. With the user locale still in English, open Word and save a document with a file name in Russian. Look at it in Explorer, and make sure it is, indeed, in Russian. Use Word's "open..." dialog to open the file. Obviously, Word can still interpret the file's name as Russian, despite having it's user locale set to English. 3. Switch your system locale to English. This will require a reboot. Open the folder containing the file in Explorer (a unicode application) - the name is still Russian. Now try to open the file in Word (a non-unicode application). It matters not if you double click the file or go through the "File/Open" dialog. The file will not open. With the system locale set to English, word has no way to tell CreateFileA to open a file who's name is outside the current codepage. This, at least, should convince you that system locale and user locale are distinct things.
Any attempt to have more than one active locale simultaneously will lead to confusion and nothing else, be it unix environment or win32.
The question is not whether confusion will arise from using different system and user locale. The question is whether it's supported, and whether we should support it as well. Please do report the results of the above test. If they are anything other than what I have said, I'll be glad to amend my understanding of Windows. Shachar -- Shachar Shemesh Lingnu Open Source Consulting ltd. http://www.lingnu.com/
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
Ok. Then please do the following experiment for me, will ya? 1. Hover the mouse over the clock. The date is displayed. Watch the date display language.
In the language of the system locale, russian for me.
Now go to "Regional options", and change the setting to "English (U.S.)". Hover over the clock again - see how the date display setting changed? Obviously, the user locale affects the date format.
That's the system locale again, and yes, when the system locale changes, date format changes as well.
2. With the user locale still in English, open Word and save a document with a file name in Russian. Look at it in Explorer, and make sure it is, indeed, in Russian. Use Word's "open..." dialog to open the file. Obviously, Word can still interpret the file's name as Russian, despite having it's user locale set to English. 3. Switch your system locale to English. This will require a reboot.
It *does not* require a reboot, that's exactly the same thing as #1 above and demonstrates exactly the same behaviour. You are trying to persuade yourself by taking into account only the look of the Windows. Please write a test app how I did and see what data GetSystemDefaultLCID and GetUserDefaultLCID return.
Open the folder containing the file in Explorer (a unicode application) - the name is still Russian. Now try to open the file in Word (a non-unicode application).
Word is a *unicode* application when it's running under NT. It uses a sophisticated system of API pointers for that task in order to run under Win9x and NT.
It matters not if you double click the file or go through the "File/Open" dialog. The file will not open. With the system locale set to English, word has no way to tell CreateFileA to open a file who's name is outside the current codepage.
This, at least, should convince you that system locale and user locale are distinct things.
I see that you do not understand, that there is no user locale at all under Windows, and therefore there is no way to change user locale. There are just adjustable per user locale data in the registry. Please realize that, otherwise it's a useless discussion, pointing again and again to the same facts.
Any attempt to have more than one active locale simultaneously will lead to confusion and nothing else, be it unix environment or win32.
The question is not whether confusion will arise from using different system and user locale. The question is whether it's supported, and whether we should support it as well.
It can't be supported in any sane manner since characters can't be mapped between incompatible charsets like hebrew and russian, and therefore characters from those charsets can't be displayed simultaneously by a not unicode application. That's exactly what you are trying to accomplish. Just realize that.
Please do report the results of the above test. If they are anything other than what I have said, I'll be glad to amend my understanding of Windows.
Your understanding of Windows is based on conclusions you made by observing visual effects, not API behaviour. -- Dmitry.
I think I found the source of our misunderstanding. To me, this is the terminology (Windows 2000): User locale (aka "User default locale"): the locale as appearing in the "Regional options" as "Your locale (location)". System locale: the locale as appearing in the "Regional options" as "Set default". Under XP, the terminology is a little better: User locale: "This option affects how some programs format numbers, currencies, dates and time" ("Regional options" tab in the control panel applet). System locale: "Language for non-Unicode programs" ("Advanced" tab in the above mentioned applet). Changing the system locale does require a reboot. If it did not require a reboot, you did not change the correct locale. If you re-run the below test, I believe you will find that there are, indeed, two distinct settings. Shachar Dmitry Timoshkov wrote:
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
Ok. Then please do the following experiment for me, will ya? 1. Hover the mouse over the clock. The date is displayed. Watch the date display language.
In the language of the system locale, russian for me.
Now go to "Regional options", and change the setting to "English (U.S.)". Hover over the clock again - see how the date display setting changed? Obviously, the user locale affects the date format.
That's the system locale again, and yes, when the system locale changes, date format changes as well.
2. With the user locale still in English, open Word and save a document with a file name in Russian. Look at it in Explorer, and make sure it is, indeed, in Russian. Use Word's "open..." dialog to open the file. Obviously, Word can still interpret the file's name as Russian, despite having it's user locale set to English. 3. Switch your system locale to English. This will require a reboot.
It *does not* require a reboot, that's exactly the same thing as #1 above and demonstrates exactly the same behaviour.
You are trying to persuade yourself by taking into account only the look of the Windows. Please write a test app how I did and see what data GetSystemDefaultLCID and GetUserDefaultLCID return.
Open the folder containing the file in Explorer (a unicode application) - the name is still Russian. Now try to open the file in Word (a non-unicode application).
Word is a *unicode* application when it's running under NT. It uses a sophisticated system of API pointers for that task in order to run under Win9x and NT.
It matters not if you double click the file or go through the "File/Open" dialog. The file will not open. With the system locale set to English, word has no way to tell CreateFileA to open a file who's name is outside the current codepage.
This, at least, should convince you that system locale and user locale are distinct things.
I see that you do not understand, that there is no user locale at all under Windows, and therefore there is no way to change user locale. There are just adjustable per user locale data in the registry. Please realize that, otherwise it's a useless discussion, pointing again and again to the same facts.
Any attempt to have more than one active locale simultaneously will lead to confusion and nothing else, be it unix environment or win32.
The question is not whether confusion will arise from using different system and user locale. The question is whether it's supported, and whether we should support it as well.
It can't be supported in any sane manner since characters can't be mapped between incompatible charsets like hebrew and russian, and therefore characters from those charsets can't be displayed simultaneously by a not unicode application. That's exactly what you are trying to accomplish. Just realize that.
Please do report the results of the above test. If they are anything other than what I have said, I'll be glad to amend my understanding of Windows.
Your understanding of Windows is based on conclusions you made by observing visual effects, not API behaviour.
-- Shachar Shemesh Lingnu Open Source Consulting ltd. http://www.lingnu.com/
Dmitry Timoshkov wrote:
You are trying to persuade yourself by taking into account only the look of the Windows. Please write a test app how I did and see what data GetSystemDefaultLCID and GetUserDefaultLCID return.
Just did: GetUserDefaultLCID: 0409 GetSystemDefaultLCID: 040d
Open the folder containing the file in Explorer (a unicode application) - the name is still Russian. Now try to open the file in Word (a non-unicode application).
Word is a *unicode* application when it's running under NT. It uses a sophisticated system of API pointers for that task in order to run under Win9x and NT.
No, it is not. Well, it's more complicated than that, obviously. "Unicode" and "ANSI" applications are just compile time macros, as you well know. An application can easily use both sets of APIs, by calling "ExtTextOutA" or "ExtTextOutW" explicitly. As far as Word processing goes, Word is obviously a Unicode application. There is no other way for it to display Hebrew and Russian in the same document. It does not rely on system APIs for that (as the fact BiDi works there better than it does on Wine shows). For display purposes, I guess it does exactly what you say it does - call ExtTextOutA on Win9x, and delay link and call ExtTextOutW on NT. There is simply no other way (as you correctly state) for it to display mixed languages. However, whenever using APIs for internal purposes is concerned, Word is an ANSI application. It does call CreateFileA, and does fail on (at least Hebrew) file names that are outside the system default locale (as I call it, explained in my previous email).
The question is not whether confusion will arise from using different system and user locale. The question is whether it's supported, and whether we should support it as well.
It can't be supported in any sane manner since characters can't be mapped between incompatible charsets like hebrew and russian,
True, but not always relevant.
and therefore characters from those charsets can't be displayed simultaneously by a not unicode application.
Again, true, but not everyone need to display stuff. Also, if I select a Hebrew codepage (system locale), I'm implying I don't want non-unicode apps to display Russian.
That's exactly what you are trying to accomplish. Just realize that.
No, it's not. The reason my personal computer is configured as it is is because I need to access Hebrew from ANSI applications, but having the user locale (which you call the system locale) set to Hebrew has some undesirable side effects. I therefor have this mixed setting. No problems for me, as I'm not using any characters that belong to the English locale that are outside the ASCII range.
Please do report the results of the above test. If they are anything other than what I have said, I'll be glad to amend my understanding of Windows.
Your understanding of Windows is based on conclusions you made by observing visual effects, not API behaviour.
See beginning of this email for contradicting evidence. Don't take my word for it, however. Even if you don't want to touch the settings as explained in my previous email, set what you call the "system locale" (and I call the "user locale") to English, and run the API test again. I believe you will see that the two differ. Shachar -- Shachar Shemesh Lingnu Open Source Consulting ltd. http://www.lingnu.com/
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
You are trying to persuade yourself by taking into account only the look of the Windows. Please write a test app how I did and see what data GetSystemDefaultLCID and GetUserDefaultLCID return.
Just did: GetUserDefaultLCID: 0409 GetSystemDefaultLCID: 040d
What exactly did you do and under what Windows flavour? I assume that was XP, can you reproduce the same thing under win9x, NT4 and win2k? I can't reproduce that neither under Windows95/98 nor under Windows2000. If that's an XP specific thing, I'd simply ignore it like an inadvertent brokeness introduced by an attempt to support more than 1 logged in user. In that case simply assume that GetUserDefaultLCID returns system locale local to the current user. That fact changes nothing for me.
Open the folder containing the file in Explorer (a unicode application) - the name is still Russian. Now try to open the file in Word (a non-unicode application).
Word is a *unicode* application when it's running under NT. It uses a sophisticated system of API pointers for that task in order to run under Win9x and NT.
No, it is not. Well, it's more complicated than that, obviously. "Unicode" and "ANSI" applications are just compile time macros, as you well know. An application can easily use both sets of APIs, by calling "ExtTextOutA" or "ExtTextOutW" explicitly.
As far as Word processing goes, Word is obviously a Unicode application. There is no other way for it to display Hebrew and Russian in the same document. It does not rely on system APIs for that (as the fact BiDi works there better than it does on Wine shows). For display purposes, I guess it does exactly what you say it does - call ExtTextOutA on Win9x, and delay link and call ExtTextOutW on NT. There is simply no other way (as you correctly state) for it to display mixed languages.
However, whenever using APIs for internal purposes is concerned, Word is an ANSI application. It does call CreateFileA, and does fail on (at least Hebrew) file names that are outside the system default locale (as I call it, explained in my previous email).
Again, you are just guessing. Run winword.exe under API tracer under NT and analyse the logs. You will see that it does not call A version of win32 APIs at all.
The question is not whether confusion will arise from using different system and user locale. The question is whether it's supported, and whether we should support it as well.
It can't be supported in any sane manner since characters can't be mapped between incompatible charsets like hebrew and russian,
True, but not always relevant.
It's always relevant.
and therefore characters from those charsets can't be displayed simultaneously by a not unicode application.
Again, true, but not everyone need to display stuff. Also, if I select a Hebrew codepage (system locale), I'm implying I don't want non-unicode apps to display Russian.
Just replace en_US in your locale settings by ru_RU leaving hebrew settings untouched and you will see.
That's exactly what you are trying to accomplish. Just realize that.
No, it's not. The reason my personal computer is configured as it is is because I need to access Hebrew from ANSI applications, but having the user locale (which you call the system locale) set to Hebrew has some undesirable side effects.
What exactly side effects do you mean here?
I therefor have this mixed setting. No problems for me, as I'm not using any characters that belong to the English locale that are outside the ASCII range.
Yes, that's exactly the problem I'm talking about. -- Dmitry.
Dmitry Timoshkov wrote:
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
You are trying to persuade yourself by taking into account only the look of the Windows. Please write a test app how I did and see what data GetSystemDefaultLCID and GetUserDefaultLCID return.
Just did: GetUserDefaultLCID: 0409 GetSystemDefaultLCID: 040d
What exactly did you do and under what Windows flavour?
Windows 2000. I detailed exactly what I did in a previous email: Assuming your locale (the only one you will acknowledge) is still set to English, run your test again. Next, set your locale to Russian, and as administrator (or a member of the administrators group), go to "settings", "control panel", and run "regional options". In the "General" tab, click on "Set default". In the new dialog box, select "English US" as the locale. Simply put, what you select at the top of the "General" tab of the Regional options control panel applet is what is returned by "GetUserDefaultLCID". Hence, I've been calling it "User locale" throughout this conversation. It is a per-user setting, and does not require restart to change. What you select when you press "Set default" is what is returned by "GetSystemDefaultLCID". Hence I've been calling it "System locale". It is a global setting, requires administrator to change, and requires a reboot to get into effect. I'm not sure what you called "system locale", or why.
I assume that was XP,
You assume wrong. In any case, the only difference between XP and W2K in that regard is that, under XP, they changed the titles of the settings in the control panel to describe what the settings actually do. They otherwise behave exactly the same, and affect the same APIs and behaviors.
can you reproduce the same thing under win9x, NT4 and win2k?
That WAS on Windows 2000. I haven't tried NT4 or Win9x, and I don't care to.
I can't reproduce that neither under Windows95/98 nor under Windows2000.
That's because you refuse to acknowledge the existence of the setting mentioned above.
That fact changes nothing for me.
I have had enough of this conversation. Shachar -- Shachar Shemesh Lingnu Open Source Consulting ltd. http://www.lingnu.com/
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
Just did: GetUserDefaultLCID: 0409 GetSystemDefaultLCID: 040d
Ok, I have reproduced it now, my apologies for previous misunderstanding. But still, it really changes nothing. System default locale defines current ANSI code page, while user default locale simply puts the overrides to the registry for numbers, currency, date/time. I still want your patch to be removed until you at least write test cases showing what exactly APIs are affected by system/user locale. Using LC_CTYPE for the system default locale (current ANSI code page) is very dubious choice as well. The whole purpose and the patch itself are pure hacks aimed to fix unexplaned side effects. Alexandre, please revert the patch until at least a test case is written and a better explanation provided what exactly wrong with locale controled by only LC_ALL and LANG. -- Dmitry.
"Dmitry Timoshkov" <dmitry(a)baikal.ru> writes:
I still want your patch to be removed until you at least write test cases showing what exactly APIs are affected by system/user locale. Using LC_CTYPE for the system default locale (current ANSI code page) is very dubious choice as well. The whole purpose and the patch itself are pure hacks aimed to fix unexplaned side effects.
Alexandre, please revert the patch until at least a test case is written and a better explanation provided what exactly wrong with locale controled by only LC_ALL and LANG.
I'm afraid I still don't see what's wrong with the patch. Obviously there can be different system and user locales on Windows, since there are APIs for that, and LC_CTYPE looks like a pretty good match for what the system locale does; I don't really understand why you are so violently opposed to that patch. Could you please explain exactly what you think this is going to break? -- Alexandre Julliard julliard(a)winehq.org
"Alexandre Julliard" <julliard(a)winehq.org> wrote:
I'm afraid I still don't see what's wrong with the patch. Obviously there can be different system and user locales on Windows, since there are APIs for that, and LC_CTYPE looks like a pretty good match for what the system locale does; I don't really understand why you are so violently opposed to that patch. Could you please explain exactly what you think this is going to break?
It breaks every user's setup where LANG and LC_CTYPE inadvertently point to different locales (like LC_CTYPE=en_US, LANG=ru_RU) and users in that case have completely not working Wine: neither keyboard input, not localized resources and everything else. I, personally, fixed several similar cases for our customers. -- Dmitry.
"Dmitry Timoshkov" <dmitry(a)baikal.ru> writes:
It breaks every user's setup where LANG and LC_CTYPE inadvertently point to different locales (like LC_CTYPE=en_US, LANG=ru_RU) and users in that case have completely not working Wine: neither keyboard input, not localized resources and everything else. I, personally, fixed several similar cases for our customers.
I'm still not clear on what you are objecting to. Changing the system locale is not going to affect keyboard input or resources. Keyboard input will be affected because we now set the Unix codepage from LC_CTYPE too, is that your main objection? Or are you saying that we shouldn't even attempt to take into account the locale config because users get it wrong anyway? -- Alexandre Julliard julliard(a)winehq.org
"Alexandre Julliard" <julliard(a)winehq.org> wrote:
I'm still not clear on what you are objecting to. Changing the system locale is not going to affect keyboard input or resources. Keyboard input will be affected because we now set the Unix codepage from LC_CTYPE too, is that your main objection?
Yes, that's the reason why LC_CTYPE and LC_MESSAGES were removed from the list of environment variables controlling Wine locale.
Or are you saying that we shouldn't even attempt to take into account the locale config because users get it wrong anyway?
We should not complicate things, especially since it's not clear at all what APIs are affected by system/user locale differences. -- Dmitry.
"Dmitry Timoshkov" <dmitry(a)baikal.ru> writes:
Yes, that's the reason why LC_CTYPE and LC_MESSAGES were removed from the list of environment variables controlling Wine locale.
I think the problem was more that we were setting both user and system locales, and for that we can really only take LC_ALL into account. But it seems reasonable to use LC_CTYPE to only change the codepage, since that's pretty much the same effect it will have under Unix too.
We should not complicate things, especially since it's not clear at all what APIs are affected by system/user locale differences.
But the fact is that there are two different locales that can be configured independently on Windows, and some users need that ability. So how do you propose to solve the issue? We could of course make it configurable in the config file instead, but it seems to me that reusing the existing Unix setup is better than requiring users to configure the same thing in two different places. -- Alexandre Julliard julliard(a)winehq.org
"Alexandre Julliard" <julliard(a)winehq.org> wrote:
We should not complicate things, especially since it's not clear at all what APIs are affected by system/user locale differences.
But the fact is that there are two different locales that can be configured independently on Windows, and some users need that ability. So how do you propose to solve the issue?
We could of course make it configurable in the config file instead, but it seems to me that reusing the existing Unix setup is better than requiring users to configure the same thing in two different places.
I like the idea of moving that setting to the config file. We can't use existing unix locale settings except LC_ALL and LANG because every user's system might have (and does have) very different locale settings, we can't assume that everyone out there configures locale in the same way. -- Dmitry.
"Dmitry Timoshkov" <dmitry(a)baikal.ru> writes:
I like the idea of moving that setting to the config file. We can't use existing unix locale settings except LC_ALL and LANG because every user's system might have (and does have) very different locale settings, we can't assume that everyone out there configures locale in the same way.
I don't see how the settings would be different, surely LC_CTYPE is always going to control the ASCII->Unicode mapping on Unix, so why shouldn't it do that on Wine? If the issue is that users change their setup without understanding the results, then surely adding even more config parameters that they need to get right is not going to improve the situation. -- Alexandre Julliard julliard(a)winehq.org
"Alexandre Julliard" <julliard(a)winehq.org> wrote:
I don't see how the settings would be different, surely LC_CTYPE is always going to control the ASCII->Unicode mapping on Unix, so why shouldn't it do that on Wine? If the issue is that users change their setup without understanding the results, then surely adding even more config parameters that they need to get right is not going to improve the situation.
The real question is whether setup with different system and user locale is really used under Windows, and if yes, what exactly purpose is pursued? In the initial patch Shachar wanted to have English user interface while being able to not break current ANSI codepage support. Instead of implementing MUI for the first part (English user interface) he proposed a simple hack to workaround the problem. I still ask to show the tests demonstrating what APIs are affected by system and user locale and fix all the places in Wine which behave differently first. Then Shachar might want to write an explanations for the users and add them to documentation how the setup with different system/user locale is supposed to behave. And only after that we can return to discussion what environment variables or config file settings will control such things. -- Dmitry.
On Wed, Jul 28, 2004 at 10:13:15PM -0700, Alexandre Julliard wrote:
"Dmitry Timoshkov" <dmitry(a)baikal.ru> writes:
I like the idea of moving that setting to the config file. We can't use existing unix locale settings except LC_ALL and LANG because every user's system might have (and does have) very different locale settings, we can't assume that everyone out there configures locale in the same way.
I don't see how the settings would be different, surely LC_CTYPE is always going to control the ASCII->Unicode mapping on Unix, so why shouldn't it do that on Wine? If the issue is that users change their setup without understanding the results, then surely adding even more config parameters that they need to get right is not going to improve the situation.
Actually, there are a number of different locale-related things that Wine needs to keep track of: 1. ANSI->Unicode translation for programs that use the ANSI calls, as has been discussed in this thread. 2. Unicode->codepage translation on standard output, and codepage->Unicode translation on standard input. Note that I could set LANG to 'en_US.UTF-16' on my Linux system, and programs SHOULD accept this. Most don't, however. 3. Unicode->codepage->Unicode translation on Linux kernels before 2.4, whereafter filenames are SUPPOSED TO be in UTF-8, and kernel modules do translation for filesystems where filenames are stored in some other charset. (OPTIONAL, as filenames are not a big deal and the newer kernel fixes it--however, there has to be a converson from the short-per-character format to UTF-8). 4. Selection of approriate language for strings in programs that use such selection, as well as time, numeric, and string formats. This is all through GetLocaleInfo(), whose first argument is an LCID returned by either GetUserDefaultLocale or GetSystemDefaultLocale. 5. The MultiByteToWideChar() and WideCharToMultiByte() functions, which allow a program to do its own conversion to and from Unicode with a specified codepage. I think (1) should be specified on a per-program basis in the config file, with a system default there, and, as final default, raw translation for ANSI-to-Unicode and something reasonable the other way. I said in another message that codepages are deprecated; I meant that the ANSI calls (as opposed to (5)) are deprecated for internationalized applications. The '.codepage' suffix of LANG and LC_CTYPE should both be searched for the answer to (2). As for graphical output to X, it doesn't seem like that should be restricted by setting LANG. For (3) there should be an option in the config file like "filesystem_codepage", but it should default to utf8. For (4), Wine should select an appropriate LangID and LCID based on the la_CC tag and return them, respectively, in response to Get*DefaultLangID and Get*LCID. In wine, at present, there is not really a seperate 'system' level. Furthermore, wine could respond to different groups of GetLocaleInfo() constants according to LC_MESSAGES, LC_NUMERIC, etc., but this is an unusual feature that probably isn't needed at first. It seems that using the config-file to define codepage translation and the suffix for IO charset translation gets rid of the typical user's need to have other variables besides LANG set. Consider locales I might use: LANG LCID LangID ---- ---- ------ en_US 1 9 es_MX 52 10 es_US 1 9 ar_SA 966 1 Let's say I have a program that prints "Hello, World" in the current language, using wide calls. When I run it in Linux, it should print that string out using the current language and codepage. Suppose I also have a database program that was written in outer burgoslavia and keeps its data files in the encoding for outer burgoslavian, which is supported only by Windows 95 for Burgoslavia and Windows Server 2003. I don't want to change Linux to support Burgoslavian, but if Burgoslavian is encoded in some Unicode font I can add a section to [AppDefaults] and let that perticular program think it is running on an all-Burgoslavian system. For (5), the functions act the same no matter what locale the user is in. -- resume at http://www.cse.msu.edu/~lamber45/resume.htm PGP key at http://www.cse.msu.edu/~lamber45/newmail.htm#GPGKey
David Lee Lambert wrote:
On Wed, Jul 28, 2004 at 10:13:15PM -0700, Alexandre Julliard wrote:
"Dmitry Timoshkov" <dmitry(a)baikal.ru> writes:
I like the idea of moving that setting to the config file. We can't use existing unix locale settings except LC_ALL and LANG because every user's system might have (and does have) very different locale settings, we can't assume that everyone out there configures locale in the same way.
I don't see how the settings would be different, surely LC_CTYPE is always going to control the ASCII->Unicode mapping on Unix, so why shouldn't it do that on Wine? If the issue is that users change their setup without understanding the results, then surely adding even more config parameters that they need to get right is not going to improve the situation.
Actually, there are a number of different locale-related things that Wine needs to keep track of:
1. ANSI->Unicode translation for programs that use the ANSI calls, as has been discussed in this thread.
Ok.
2. Unicode->codepage translation on standard output, and codepage->Unicode translation on standard input. Note that I could set LANG to 'en_US.UTF-16' on my Linux system, and programs SHOULD accept this. Most don't, however.
Why should we set it differently than 1? In any case, I am not aware of UTF-16 being a compilable locale setting. Thus, it is not required that anyone support it.
3. Unicode->codepage->Unicode translation on Linux kernels before 2.4, whereafter filenames are SUPPOSED TO be in UTF-8, and kernel modules do translation for filesystems where filenames are stored in some other charset. (OPTIONAL, as filenames are not a big deal and the newer kernel fixes it--however, there has to be a converson from the short-per-character format to UTF-8).
Name one such filesystem, please. EXT and reiser never cared, as far as I know. VFAT has to translate names stored in UTF-16. Are you saying the kernels<2.4 didn't have the "iocharset" option?
4. Selection of approriate language for strings in programs that use such selection,
Discussed in this thread under the "GetDefaultUILanguage" API.
as well as time, numeric, and string formats.
Also discussed in this thread.
This is all through GetLocaleInfo(), whose first argument is an LCID returned by either GetUserDefaultLocale or GetSystemDefaultLocale.
You can also pass "LOCALE_SYSTEM_DEFAULT" instead, but that doesn't matter. In any case, there are "user overrides" here, which we may, one day, want to implement. Everything is laid out in the table that started this thread.
5. The MultiByteToWideChar() and WideCharToMultiByte() functions, which allow a program to do its own conversion to and from Unicode with a specified codepage.
What do we need to do with these? They get an explicit codepage to convert to/from. Funny though it may sound, these functions are not affected by locale.
I think (1) should be specified on a per-program basis in the config file, with a system default there, and, as final default, raw translation for ANSI-to-Unicode and something reasonable the other way. I said in another message that codepages are deprecated; I meant that the ANSI calls (as opposed to (5)) are deprecated for internationalized applications.
I don't agree. Mixing default codepages across simultaneously running programs is not possible on Windows, and sounds horribly difficult to implement. Clipboard handling and cross-file using are two examples of things that are likely to go horribly wrong if we tried. Having one setting applicable to all running processes sounds good enough. I don't object to a config setting overriding what LC_CTYPE says, but I don't see a use for it either.
The '.codepage' suffix of LANG and LC_CTYPE should both be searched for the answer to (2). As for graphical output to X, it doesn't seem like that should be restricted by setting LANG.
Again - why should it be different than 1?
For (3) there should be an option in the config file like "filesystem_codepage", but it should default to utf8.
We should probably not bother, though. This "problem" is shared by every other Unix program running on the system, and solved the same way there - they use LC_CTYPE.
For (4), Wine should select an appropriate LangID and LCID based on the la_CC tag and return them, respectively, in response to Get*DefaultLangID and Get*LCID. In wine, at present, there is not really a seperate 'system' level.
Furthermore, wine could respond to different groups of GetLocaleInfo() constants according to LC_MESSAGES, LC_NUMERIC, etc., but this is an unusual feature that probably isn't needed at first. It seems that using the config-file to define codepage translation and the suffix for IO charset translation gets rid of the typical user's need to have other variables besides LANG set.
Consider locales I might use:
LANG LCID LangID ---- ---- ------ en_US 1 9 es_MX 52 10 es_US 1 9 ar_SA 966 1
Let's say I have a program that prints "Hello, World" in the current language, using wide calls. When I run it in Linux, it should print that string out using the current language and codepage. Suppose I also have a database program that was written in outer burgoslavia and keeps its data files in the encoding for outer burgoslavian, which is supported only by Windows 95 for Burgoslavia and Windows Server 2003. I don't want to change Linux to support Burgoslavian, but if Burgoslavian is encoded in some Unicode font I can add a section to [AppDefaults] and let that perticular program think it is running on an all-Burgoslavian system.
For (5), the functions act the same no matter what locale the user is in.
-- Shachar Shemesh Lingnu Open Source Consulting ltd. http://www.lingnu.com/
On Wed, Aug 04, 2004 at 11:20:57AM +0300, Shachar Shemesh wrote:
David Lee Lambert wrote:
On Wed, Jul 28, 2004 at 10:13:15PM -0700, Alexandre Julliard wrote:
"Dmitry Timoshkov" <dmitry(a)baikal.ru> writes:
I don't see how the settings would be different, surely LC_CTYPE is always going to control the ASCII->Unicode mapping on Unix, so why shouldn't it do that on Wine? If the issue is that users change their setup without understanding the results, then surely adding even more config parameters that they need to get right is not going to improve the situation.
1. ANSI->Unicode translation for programs that use the ANSI calls, as has been discussed in this thread.
Ok.
2. Unicode->codepage translation on standard output, and codepage->Unicode translation on standard input. ...
Why should we set it differently than 1?
(1) is when a program talks to what it thinks is a version of Windows according to the author's understanding of a the codepage. (2) and (3) are how are how Wine talks to the rest of the system. What codepage should be used under a utf8 locale, such as most of the setups on Redhat WS 9? There are no Windows programs that actually pass utf8 data to the Xxxx*A calls; if a program wants to use Unicode, it will use the wide character APIs. (MS Word, PAF 5, IE 6...) (I know I said UTF16 before. I haven't actually tried a UTF16-only setup, and UTF16 is not compatible with the C library because it uses \0 in normal text, etc.)
Name one such filesystem, please. EXT and reiser never cared, as far as I know. VFAT has to translate names stored in UTF-16. Are you saying the kernels<2.4 didn't have the "iocharset" option?
I have some files stored on a Windows ME box with names containing accented characters, which show up fine in Explorer and the MS-DOS prompt on that system and on other WinME systems accross the network. In Linux the filenames appear to be encoded in CP437 when I use 'smbmount' whith no options. The manpage says that iocharset= (Linux side) and codepage= (Server side) are only supported in kernel 2.4 and above. However, on my system that runs kernel 2.4.x, the problem persists even when I pass options "iocharset=utf8,codepage=cp437" I get a message about CP437 not being supported, even though I have a kernel module cp437.o . By the way, Explorer and command.com only allow me to enter directories with 8-bit-character-names on the local system. Under Linux, I can enter said directory using tab completion, even though the characters don't show up.
This is all through GetLocaleInfo(), whose first argument is an LCID returned by either GetUserDefaultLocale or GetSystemDefaultLocale.
You can also pass "LOCALE_SYSTEM_DEFAULT" instead, but that doesn't matter. In any case, there are "user overrides" here, which we may, one day, want to implement. Everything is laid out in the table that started this thread.
5. The MultiByteToWideChar() and WideCharToMultiByte() functions, which
What do we need to do with these? They get an explicit codepage to convert to/from. Funny though it may sound, these functions are not affected by locale.
Right.
I don't agree. Mixing default codepages across simultaneously running programs is not possible on Windows, and sounds horribly difficult to implement. Clipboard handling and cross-file using are two examples of things that are likely to go horribly wrong if we tried.
If a program does a an ANSI call, it gets changed to Unicode. If that data gets passed back to another program using an ANSI call, it gets converted to the target codepage, as much as possible. If Wine is doing console output to a non-Unicode codepage, it gets converted there too. I'm not sure about mixing codepages under Windows, but input-method switching is easy.
Having one setting applicable to all running processes sounds good enough. I don't object to a config setting overriding what LC_CTYPE says, but I don't see a use for it either.
Let's say I want to run an Arabic dictionary program and Spanish dictionary program as I'm typing a report in Word. Furthermore, the arabic dictionary program would use ANSI calls and expects to run on Arabic windows (with MS-ARAB encoding); the Spanish program would use LATIN1. Word uses wide-character calls. How would I run all these programs at the same time under Wine? (I'm not saying I actually have such a set of programs. Actually, I own a copy of a standalone program that allows typing of Arabic programs on Western windows, and also have access to systems where Office XP with the Arabic and US-International input methods is installed.) -- resume at http://www.cse.msu.edu/~lamber45/resume.htm PGP key at http://www.cse.msu.edu/~lamber45/newmail.htm#GPGKey
David Lee Lambert wrote:
I don't agree. Mixing default codepages across simultaneously running programs is not possible on Windows, and sounds horribly difficult to implement. Clipboard handling and cross-file using are two examples of things that are likely to go horribly wrong if we tried.
If a program does a an ANSI call, it gets changed to Unicode. If that data gets passed back to another program using an ANSI call, it gets converted to the target codepage, as much as possible. If Wine is doing console output to a non-Unicode codepage, it gets converted there too.
Ok, you convinced me. Implement that in a way that Alexandre will deem fit for commit, and I won't object too loudly. I do believe that you will not succeed with the first part, however.
I'm not sure about mixing codepages under Windows, but input-method switching is easy.
True, but irrelevant.
Having one setting applicable to all running processes sounds good enough. I don't object to a config setting overriding what LC_CTYPE says, but I don't see a use for it either.
Let's say I want to run an Arabic dictionary program and Spanish dictionary program as I'm typing a report in Word. Furthermore, the arabic dictionary program would use ANSI calls and expects to run on Arabic windows (with MS-ARAB encoding); the Spanish program would use LATIN1. Word uses wide-character calls. How would I run all these programs at the same time under Wine?
Forget that. How would you run that under Windows? As far as I know, you can't do it there either. Coming from a country where a large portion of the population are native Russian speakers in a country where the dominant language is Hebrew, I'm pretty sure that I'd hear about it if it were possible.
(I'm not saying I actually have such a set of programs. Actually, I own a copy of a standalone program that allows typing of Arabic programs on Western windows, and also have access to systems where Office XP with the Arabic and US-International input methods is installed.)
Yes. That's because they either use the Unicode interface, or implement their own rendering engine. In short, I think what you're trying is impossible under Windows. I can also see some immediate implications to trying to do it in Wine. If it's important to you, you're welcome at it. It will not enter my personal todo list, however. Shachar -- Shachar Shemesh Lingnu Open Source Consulting ltd. http://www.lingnu.com/
Dmitry Timoshkov wrote:
We should not complicate things, especially since it's not clear at all what APIs are affected by system/user locale differences.
System locale affects - String comparsions (IntlStrEqWorker[AW], lstrcmp[AW], lstrcmpi[AW]) Uses system locale after trying thread locale (which is derived from user locale on thread creation). - Resource loading (FindResource & friends) Uses system locale after trying Default UI Language (see GetSystemDefaultUILanguage and GetUserDefaultUILanguage or NtQueryDefaultLanguage) and user locale. - Input locale identifier The thread default input language for system applications (that are started before user logins) is derived from the system locale setting. This probably isn't relevant for Wine. And if you ask, no, I will not write a testcase, because it's sometimes hard for me to even understand the problems caused by different locales. Regards, Filip
Filip Navara wrote:
Dmitry Timoshkov wrote:
We should not complicate things, especially since it's not clear at all what APIs are affected by system/user locale differences.
System locale affects
System locale affects all *A functions. I.e. - it affects all ANSI functions. Aside from this affect:
- String comparsions (IntlStrEqWorker[AW], lstrcmp[AW], lstrcmpi[AW]) Uses system locale after trying thread locale (which is derived from user locale on thread creation).
Uses User locale. Not system locale. It determines the collation (sort order). Obviously, it affects the A functions (lstrcmpA, etc.), but no more nor less than it does CreateWindowA, FormatHardDiskA, or AnyOtherFunctionA.
- Resource loading (FindResource & friends) Uses system locale after trying Default UI Language (see GetSystemDefaultUILanguage and GetUserDefaultUILanguage or NtQueryDefaultLanguage) and user locale.
Actually, here I don't think system locale affects at all. I'll be able to tell with more certainty later, though.
- Input locale identifier The thread default input language for system applications (that are started before user logins) is derived from the system locale setting. This probably isn't relevant for Wine.
And if you ask, no, I will not write a testcase, because it's sometimes hard for me to even understand the problems caused by different locales.
Regards, Filip
Shachar -- Shachar Shemesh Lingnu Open Source Consulting ltd. http://www.lingnu.com/
Dmitry Timoshkov wrote:
"Alexandre Julliard" <julliard(a)winehq.org> wrote:
I'm afraid I still don't see what's wrong with the patch. Obviously there can be different system and user locales on Windows, since there are APIs for that, and LC_CTYPE looks like a pretty good match for what the system locale does; I don't really understand why you are so violently opposed to that patch. Could you please explain exactly what you think this is going to break?
It breaks every user's setup where LANG and LC_CTYPE inadvertently point to different locales (like LC_CTYPE=en_US, LANG=ru_RU) and users in that case have completely not working Wine: neither keyboard input, not localized resources and everything else. I, personally, fixed several similar cases for our customers.
What happens on Unix with that setup? When I try setting LANG to he_IL and LC_CTYPE to en_US on unix, I can type Hebrew characters in kedit, but they are saved as "?". In other words, Wine's behavior under the case you mention exactly reflects the behavior Unix gives. I think the GIGO rule should apply here. Garbage in, garbage out. Shachar -- Shachar Shemesh Lingnu Open Source Consulting ltd. http://www.lingnu.com/
Shachar Shemesh wrote:
What happens on Unix with that setup? When I try setting LANG to he_IL and LC_CTYPE to en_US on unix, I can type Hebrew characters in kedit, but they are saved as "?". In other words, Wine's behavior under the case you mention exactly reflects the behavior Unix gives.
I think the GIGO rule should apply here. Garbage in, garbage out.
Shachar
It seems that Wine under the above circumstances does not allow the keyboard input at all, not even for Unicode apps. This, however, strikes me as a bug in the keyboard input area, not in the locale handling area. Shachar -- Shachar Shemesh Lingnu Open Source Consulting ltd. http://www.lingnu.com/
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
What happens on Unix with that setup? When I try setting LANG to he_IL and LC_CTYPE to en_US on unix, I can type Hebrew characters in kedit, but they are saved as "?". In other words, Wine's behavior under the case you mention exactly reflects the behavior Unix gives.
I think the GIGO rule should apply here. Garbage in, garbage out.
Shachar
It seems that Wine under the above circumstances does not allow the keyboard input at all, not even for Unicode apps. This, however, strikes me as a bug in the keyboard input area, not in the locale handling area.
That's a bug in X11 which does not have a unicode input system and has no such a thing as input locale. You always can use 'xev' to see what keyboard inout X11 sends to an application. You will see that Wine has nothing to do with that. -- Dmitry.
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
What happens on Unix with that setup? When I try setting LANG to he_IL and LC_CTYPE to en_US on unix, I can type Hebrew characters in kedit,
KDE does not use raw X11 protocol as Wine does. Run 'xev' and try to enter hebrew characters with LC_CTYPE=en_US.
but they are saved as "?". In other words, Wine's behavior under the case you mention exactly reflects the behavior Unix gives.
That's not a Wine problem, that's a problem of the broken user locale. -- Dmitry.
Dmitry Timoshkov wrote:
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
but they are saved as "?". In other words, Wine's behavior under the case you mention exactly reflects the behavior Unix gives.
That's not a Wine problem, that's a problem of the broken user locale.
But that's exactly my point. Wine's behavior correctly reflects that of the user locale. Wine does not break here, it's the user locale that was broken to begin with. It did not work in Unix either. Ergo - there is no problem with wine. You are effectively saying here that you want Wine to second guess the user's locale, and fix broken setups. Well, MS has more than 100 times the amount of developers we have, and any time they went with that methodology, their product got just a little more broken (IE's mime second guessing is a prime example). I don't think we should bring that insanity into Wine, thank you. If the user's locale is broken, it's perfectly ok in my book for Wine to be broken too. Picking a random environment variable and saying "We will use this one and not that one" is utter nonsense. Just so things are clear, picking LC_ALL and LANG for the system locale, but not LC_CTYPE is precisely that - picking random variables as representing of functionality typically provided by other variables. The flip side of your argument regarding broken locales now being broken on Wine as well is that, if this patch is reverted, perfectly valid locales (such as mine - LANG=en_US, LC_CTYPE=he_IL) are broken. Now which do you prefer? Broken setups not being "fixed", or valid setups being broken? Shachar -- Shachar Shemesh Lingnu Open Source Consulting ltd. http://www.lingnu.com/
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
If the user's locale is broken, it's perfectly ok in my book for Wine to be broken too. Picking a random environment variable and saying "We will use this one and not that one" is utter nonsense. Just so things are clear, picking LC_ALL and LANG for the system locale, but not LC_CTYPE is precisely that - picking random variables as representing of functionality typically provided by other variables.
LC_ALL and LANG are not random variables, that's what glibc uses for internal locale setup.
The flip side of your argument regarding broken locales now being broken on Wine as well is that, if this patch is reverted, perfectly valid locales (such as mine - LANG=en_US, LC_CTYPE=he_IL) are broken.
Your locale is broken, and I already explained why. You are lucky that en_US has no conflicting with he_IL characters in the upper ASCII table. Just replace en_US by fr_FR, de_DE or ru_RU. Why are you ignoring this fact?
Now which do you prefer? Broken setups not being "fixed", or valid setups being broken?
I prefer not introduce potentially confusing things. I'm still waiting for the test cases and explanations what exactly APIs are affected by system/user locale in Windows and what users should expect when they have different system and user locale. Until you answer that questions I don't see how you would argue that your patch is correct and does exactly the same thing as supposed to happened under Windows. -- Dmitry.
Dmitry Timoshkov wrote:
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
If the user's locale is broken, it's perfectly ok in my book for Wine to be broken too. Picking a random environment variable and saying "We will use this one and not that one" is utter nonsense. Just so things are clear, picking LC_ALL and LANG for the system locale, but not LC_CTYPE is precisely that - picking random variables as representing of functionality typically provided by other variables.
LC_ALL and LANG are not random variables, that's what glibc uses for internal locale setup.
Ignoring LC_CTYPE is as random as ignoring any other one. For the 100th time, we are not ignoring LC_ALL and LANG. We are simply ALSO looking at LC_CTYPE.
The flip side of your argument regarding broken locales now being broken on Wine as well is that, if this patch is reverted, perfectly valid locales (such as mine - LANG=en_US, LC_CTYPE=he_IL) are broken.
Your locale is broken, and I already explained why. You are lucky that en_US has no conflicting with he_IL characters in the upper ASCII table. Just replace en_US by fr_FR, de_DE or ru_RU. Why are you ignoring this fact?
Because it's irrelevant whether there are broken locales that have LC_CTYPE different than LANG. The right question is whether there are NON broken locales of that type. Luck has nothing to do with it. If the setup I'm using was not working, I wouldn't be using it. Another interesting question is whether there are non-broken Unix locales that result in a broken Wine locale as a result of this patch. The answers, as you correctly state in this very question, is YES and NO respectively, meaning there appears to be no base in reality for your categorical claim that ANY setup with LC_CTYPE is broken. In other words - my system is running such a locale. Why is it that you wish to break my setup? Especially since it doesn't affect your setup at all!
Now which do you prefer? Broken setups not being "fixed", or valid setups being broken?
I prefer not introduce potentially confusing things. I'm still waiting for the test cases and explanations what exactly APIs are affected by system/user locale in Windows and what users should expect when they have different system and user locale. Until you answer that questions I don't see how you would argue that your patch is correct and does exactly the same thing as supposed to happened under Windows.
We have a property called "LC_CTYPE" on Unix. We have a property called "System locale" on Windows. They are both documented and observed to do exactly the same. Mapping one to the other seems the right thing to do. There is no need to create an elaborate system of test cases just so I can prove something which appears to be right is really right. If you are so certain it's wrong, why don't you do one of two things: 1. Show us that LC_CTYPE and system locale are NOT the same thing. 2. Show us what system APIs functionality is broken by this patch. Please bear in mind that I can't think of a single case of 2 above for which the answer would be "System locale should not be affected by LC_CTYPE". I am much more likely to think "the specific API you found to break is the curl pit". I am, however, open to be shown wrong on this. The most curious thing for me is that this patch does not even affect your setup. If you didn't touch LC_CTYPE (as you claim no one should), why should you care about this patch? It will still take system locale from LANG if LC_CTYPE is not defined. In other words, this patch fixes Wine on my system without breaking it on yours. Any system on which it does break is a system you categorically call broken to begin with (LC_CTYPE points elsewhere from LANG). Why do you object so much? Shachar -- Shachar Shemesh Lingnu Open Source Consulting ltd. http://www.lingnu.com/
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
Ignoring LC_CTYPE is as random as ignoring any other one. For the 100th time, we are not ignoring LC_ALL and LANG. We are simply ALSO looking at LC_CTYPE.
We were looking for LC_CTYPE before and it was removed from the list of environment variables for a reason. And now you introduced completely different LC_CTYPE handling unlike LC_ALL and LANG do. It's not "ALSO".
We have a property called "LC_CTYPE" on Unix. We have a property called "System locale" on Windows. They are both documented and observed to do exactly the same. Mapping one to the other seems the right thing to do.
There is no need to create an elaborate system of test cases just so I can prove something which appears to be right is really right.
It only appears unfortunately.
If you are so certain it's wrong, why don't you do one of two things: 1. Show us that LC_CTYPE and system locale are NOT the same thing. 2. Show us what system APIs functionality is broken by this patch.
That's nonsense to ask me or somebody else to proof your patch is wrong. That's your responsibility to proof its correctness to everyone else, writing test cases and pointing out to documentation if necessary.
In other words, this patch fixes Wine on my system without breaking it on yours. Any system on which it does break is a system you categorically call broken to begin with (LC_CTYPE points elsewhere from LANG). Why do you object so much?
I'm doing a lot of work supporting cxoffice users fixing locale related bugs for them. Since the release 3.0 I haven't had a single locale related bug report, and therefore I assume that most of the bugs are fixed. Now you want to break that fragile balance. How can I keep silence? -- Dmitry.
Dmitry Timoshkov wrote:
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
Ignoring LC_CTYPE is as random as ignoring any other one. For the 100th time, we are not ignoring LC_ALL and LANG. We are simply ALSO looking at LC_CTYPE.
We were looking for LC_CTYPE before and it was removed from the list of environment variables for a reason.
Yes. That reason was that it was setting both user and system locale. Wrong fix, through right direction.
It only appears unfortunately.
Then please qualify your statement. Where do they diverge?
That's nonsense to ask me or somebody else to proof your patch is wrong. That's your responsibility to proof its correctness to everyone else, writing test cases and pointing out to documentation if necessary.
When I pointed to documentation, you claimed (with no supporting evidence, I might add) that system and user locale are the same, and that the documentation is wrong. You since amended your claim, no longer claiming that system and user locale are the same, but still claiming the documentation is wrong. You now ask me to write tests proving the documentation is right? I'm sorry, I'm not going to through that insanity again.
In other words, this patch fixes Wine on my system without breaking it on yours. Any system on which it does break is a system you categorically call broken to begin with (LC_CTYPE points elsewhere from LANG). Why do you object so much?
I'm doing a lot of work supporting cxoffice users fixing locale related bugs for them. Since the release 3.0 I haven't had a single locale related bug report, and therefore I assume that most of the bugs are fixed.
That is not true. I have personally seen at least one message on the cx support mailing list where a CW employee (I think it was you) explained to an Israeli user (who was not me, mind you :) that using LC_CTYPE was wrong, and that their system is broken. Their system was working perfectly with Unix applications. As I have explained before, I reject that notion. Agree me with me or don't, you cannot claim that CX has no locale related problems. Also, please bear in mind that crossover didn't support BiDi (at least, until I did http://www.lingnu.com/support.html), so a large part of the people potentially affected by this problem did not run it.
Now you want to break that fragile balance. How can I keep silence?
Dmitry, please believe me that I do not do things just so you have more work. However, you have to accept that your knowledge of the way Windows handles locales was flawed until yesterday. You also have to accept that you don't know all setups. As such, please consider the possibility that removing LC_CTYPE as something that initialize the user locale was an improvement, while introducing it as something that initialize the system locale is another improvement (rather than a set back). Now, if you want to help, please compile a short list of the locale settings that caused you grief in the past, and let's go over them and see how Wine should handle them. This will allow us to move forward in a constructive manner. Shachar
"Dmitry Timoshkov" <dmitry(a)baikal.ru> writes:
I prefer not introduce potentially confusing things. I'm still waiting for the test cases and explanations what exactly APIs are affected by system/user locale in Windows and what users should expect when they have different system and user locale.
The system locale defines the current codepages, so it affects MultiByteToWideChar and similar APIs. The user locale affects resource loading. It's likely that there are places in Wine that don't handle the distinction properly, but I don't think it's reasonable to demand tests for every single API before we can consider the patch. Particularly since without the patch there is no way to make user and system locales different, so there is no way to test anything at all. -- Alexandre Julliard julliard(a)winehq.org
Alexandre Julliard wrote:
The user locale affects resource loading.
Actually, that's a bug in Wine. User locale should not affect that. It should affect functions such as GetCurencyFormat, GetDateFormat etc. I'm now working on a fix to that problem. That, however, is unrelated to the system locale discussion, of course. Shachar
Shachar Shemesh <wine-devel(a)shemesh.biz> writes:
Actually, that's a bug in Wine. User locale should not affect that. It should affect functions such as GetCurencyFormat, GetDateFormat etc. I'm now working on a fix to that problem.
That's not true, user locale definitely affects resource loading. It should also be affected by UI language, which isn't implemented now, so that's probably the difference you see. -- Alexandre Julliard julliard(a)winehq.org
"Alexandre Julliard" <julliard(a)winehq.org> wrote:
The system locale defines the current codepages, so it affects MultiByteToWideChar and similar APIs. The user locale affects resource loading. It's likely that there are places in Wine that don't handle the distinction properly, but I don't think it's reasonable to demand tests for every single API before we can consider the patch.
There is no need to test every API, but there are APIs we need to require mandatory testing: GetSystemDefaultLCID GetUserDefaultLCID GetACP FindResourceA FindResourceExA CompareStringA GetLocaleInfoA
Particularly since without the patch there is no way to make user and system locales different, so there is no way to test anything at all.
Right, that's why I'm asking to revert patch and commit it simultaneously with a test case and possible fixes for bugs revealed by the tests. -- Dmitry.
"Dmitry Timoshkov" <dmitry(a)baikal.ru> writes:
There is no need to test every API, but there are APIs we need to require mandatory testing:
GetSystemDefaultLCID GetUserDefaultLCID GetACP
I don't see what you want to test here.
FindResourceA FindResourceExA
These ones have been well tested already.
CompareStringA GetLocaleInfoA
These take an explicit locale parameter so I don't see why changing the system locale would suddenly break them.
Right, that's why I'm asking to revert patch and commit it simultaneously with a test case and possible fixes for bugs revealed by the tests.
I see no reason to revert the patch. It's going to make no difference for 99% of our users, and for the remaining 1% it's about as likely to fix things as it is to break them. -- Alexandre Julliard julliard(a)winehq.org
"Alexandre Julliard" <julliard(a)winehq.org> wrote:
There is no need to test every API, but there are APIs we need to require mandatory testing:
GetSystemDefaultLCID GetUserDefaultLCID GetACP
I don't see what you want to test here.
I want to test whether locale has really changed and what subsequences happened. I want to see system/user locale before the switch and after. I'd like also add GetSystemDefaultLangID, GetSystemDefaultUILanguage, GetUserDefaultLangID and GetUserDefaultUILanguage to the list. Probably also GetThreadLocale.
FindResourceA FindResourceExA
These ones have been well tested already.
I don't see the tests for them in the CVS, especially part about LANGID preference order.
CompareStringA GetLocaleInfoA
These take an explicit locale parameter so I don't see why changing the system locale would suddenly break them.
You are right.
Right, that's why I'm asking to revert patch and commit it simultaneously with a test case and possible fixes for bugs revealed by the tests.
I see no reason to revert the patch. It's going to make no difference for 99% of our users, and for the remaining 1% it's about as likely to fix things as it is to break them.
OK, in the worst case perhaps I will need to send a custom version of kernel32 with the patch reverted to the affected cxoffice users. -- Dmitry.
"Dmitry Timoshkov" <dmitry(a)baikal.ru> writes:
"Alexandre Julliard" <julliard(a)winehq.org> wrote:
FindResourceA FindResourceExA
These ones have been well tested already.
I don't see the tests for them in the CVS, especially part about LANGID preference order.
There's no automated test, because it's a bit tricky to switch locales under Windows since not all locales may be supported on a given install. I can assure you that it has been tested and that the behavior is correct (modulo UI language as mentioned already).
I see no reason to revert the patch. It's going to make no difference for 99% of our users, and for the remaining 1% it's about as likely to fix things as it is to break them.
OK, in the worst case perhaps I will need to send a custom version of kernel32 with the patch reverted to the affected cxoffice users.
If they really have configured LC_CTYPE in a way that breaks things, I would suggest to them to fix it instead, this way it will also fix their Unix setup. Now, if you really know of a case where there is a legitimate reason for someone to have an LC_CTYPE setup that breaks Wine, then of course that would be a good argument against the patch. -- Alexandre Julliard julliard(a)winehq.org
Dmitry Timoshkov wrote:
"Shachar Shemesh" <wine-devel(a)shemesh.biz> wrote:
Just did: GetUserDefaultLCID: 0409 GetSystemDefaultLCID: 040d
Ok, I have reproduced it now, my apologies for previous misunderstanding. But still, it really changes nothing. System default locale defines current ANSI code page, while user default locale simply puts the overrides to the registry for numbers, currency, date/time.
I still want your patch to be removed until you at least write test cases showing what exactly APIs are affected by system/user locale.
Docs says system locale and LC_CTYPE do the same thing. My experience says system locale and LC_CTYPE do the same thing. You yourself says that system locale affects the ANSI code page, and that's what LC_CTYPE does. I see no reason for further discussions myself. If Alexandre says otherwise, I'll be glad to answer any specific concerns anyone has.
Using LC_CTYPE for the system default locale (current ANSI code page) is very dubious choice as well. The whole purpose and the patch itself are pure hacks aimed to fix unexplaned side effects.
That's not how I see it. I have spent quite enough time on this discussion to "write tests".
Alexandre, please revert the patch until at least a test case is written and a better explanation provided what exactly wrong with locale controled by only LC_ALL and LANG.
I don't see any reason to revert the patch merely because you don't like it. If you are so sure that the docs are all wrong, show us where. Shachar -- Shachar Shemesh Lingnu Open Source Consulting ltd. http://www.lingnu.com/
Dmitry Timoshkov wrote:
I still want your patch to be removed until you at least write test cases showing what exactly APIs are affected by system/user locale. Using LC_CTYPE for the system default locale (current ANSI code page) is very dubious choice as well. The whole purpose and the patch itself are pure hacks aimed to fix unexplaned side effects.
I think it would be best to set the codepage for an app in the [AppDefaults] section. Codepages are deprecated in favor of full Unicode support, but specific apps might use ANSI APIs, and might even expect certain locales. For instance, Quattro Pro 9 appears to parse text from the clipboard and from ODBC calls as CP437 on my WinME boxes. By the way, I think a system-wide /etc/wine.conf should be brought back before the 1.0 release. -- DLL
"David Lee Lambert" <lamber45(a)cse.msu.edu> wrote:
I think it would be best to set the codepage for an app in the [AppDefaults] section. Codepages are deprecated in favor of full Unicode support, but specific apps might use ANSI APIs, and might even expect certain locales. For instance, Quattro Pro 9 appears to parse text from the clipboard and from ODBC calls as CP437 on my WinME boxes.
Code pages are not deprecated, Win64 has ANSI APIs and I bet all future versions of Windows will have that support included. Until there are non-unicode .txt files and there is a need to exchange data with the world code pages will be used. -- Dmitry.
participants (5)
-
Alexandre Julliard -
David Lee Lambert -
Dmitry Timoshkov -
Filip Navara -
Shachar Shemesh