Re: [PATCH] kernel32: Respect the LANG environment variable on Mac OS.
The call to setenv() has 0 (false) as its third parameter, so that LANG is not overwritten. -Ken On Nov 23, 2009, at 9:12 AM, Charles Davis wrote:
--- dlls/kernel32/locale.c | 53 +++++++++++++++++++++++++---------------------- 1 files changed, 28 insertions(+), 25 deletions(-)
diff --git a/dlls/kernel32/locale.c b/dlls/kernel32/locale.c index fb9b517..d9ea8eb 100644 --- a/dlls/kernel32/locale.c +++ b/dlls/kernel32/locale.c @@ -2925,32 +2925,35 @@ void LOCALE_Init(void) UINT ansi_cp = 1252, oem_cp = 437, mac_cp = 10000, unix_cp;
#ifdef __APPLE__ - /* MacOS doesn't set the locale environment variables so we have to do it ourselves */ - CFArrayRef preferred_locales, all_locales; CFStringRef user_language_string_ref = NULL; - char user_locale[50]; - char* p; - - CFLocaleRef user_locale_ref = CFLocaleCopyCurrent(); - CFStringRef user_locale_string_ref = CFLocaleGetIdentifier( user_locale_ref ); - - CFStringGetCString( user_locale_string_ref, user_locale, sizeof(user_locale), kCFStringEncodingUTF8 ); - CFRelease( user_locale_ref ); - /* Strip modifiers because setlocale() can't parse them. */ - if ((p = strchr( user_locale, '@' ))) *p = 0; - if (!strchr( user_locale, '.' )) strcat( user_locale, ".UTF-8" ); - unix_cp = CP_UTF8; /* default to utf-8 even if we don't get a valid locale */ - setenv( "LANG", user_locale, 0 ); - TRACE( "setting locale to '%s'\n", user_locale ); - - /* We still want to set the retrieve the preferred language as chosen in - System Preferences.app, because it can differ from CFLocaleCopyCurrent(). - */ - all_locales = CFLocaleCopyAvailableLocaleIdentifiers(); - preferred_locales = CFBundleCopyLocalizationsForPreferences( all_locales, NULL ); - if (preferred_locales && CFArrayGetCount( preferred_locales )) - user_language_string_ref = CFArrayGetValueAtIndex( preferred_locales, 0 ); - CFRelease( all_locales ); + if (!getenv( "LANG" )) + { + /* MacOS doesn't set the locale environment variables so we have to do it ourselves */ + CFArrayRef preferred_locales, all_locales; + char user_locale[50]; + char* p; + + CFLocaleRef user_locale_ref = CFLocaleCopyCurrent(); + CFStringRef user_locale_string_ref = CFLocaleGetIdentifier( user_locale_ref ); + + CFStringGetCString( user_locale_string_ref, user_locale, sizeof(user_locale), kCFStringEncodingUTF8 ); + CFRelease( user_locale_ref ); + /* Strip modifiers because setlocale() can't parse them. */ + if ((p = strchr( user_locale, '@' ))) *p = 0; + if (!strchr( user_locale, '.' )) strcat( user_locale, ".UTF-8" ); + unix_cp = CP_UTF8; /* default to utf-8 even if we don't get a valid locale */ + setenv( "LANG", user_locale, 0 ); + TRACE( "setting locale to '%s'\n", user_locale ); + + /* We still want to set the retrieve the preferred language as chosen in + System Preferences.app, because it can differ from CFLocaleCopyCurrent(). + */ + all_locales = CFLocaleCopyAvailableLocaleIdentifiers(); + preferred_locales = CFBundleCopyLocalizationsForPreferences( all_locales, NULL ); + if (preferred_locales && CFArrayGetCount( preferred_locales )) + user_language_string_ref = CFArrayGetValueAtIndex( preferred_locales, 0 ); + CFRelease( all_locales ); + } #endif /* __APPLE__ */
setlocale( LC_ALL, "" );
Ken Thomases wrote:
The call to setenv() has 0 (false) as its third parameter, so that LANG is not overwritten. Then why does Wine not repsect the LANG setting on Mac OS X? I've seen it on wine-users. Wherever the problem lies, it lies elsewhere.
(Yeah, I forgot what that third parameter to setenv() does.)
-Ken
Chip
I just realized something. The reason Wine doesn't respect LANG is that it overrides the LC_MESSAGES setting if it is set to the default. It will be set to the default if there is no LC_MESSAGES in the environment. So we don't want to override LC_MESSAGES if there's a LANG variable in the environment. Besides, why do all that work if we're not going to override LANG anyway if it's already set?
On Nov 24, 2009, at 7:52 AM, Charles Davis wrote:
I just realized something. The reason Wine doesn't respect LANG is that it overrides the LC_MESSAGES setting if it is set to the default. It will be set to the default if there is no LC_MESSAGES in the environment. So we don't want to override LC_MESSAGES if there's a LANG variable in the environment. Besides, why do all that work if we're not going to override LANG anyway if it's already set?
Mac OS X's Language & Text (formerly known as International) settings in System Preferences don't quite map exactly to Unix LANG or LC_* environment variables, which are a poor intermediary for mapping to Windows settings. The user's preferred language corresponds to LC_MESSAGES in intent. All the other LC_* variables correspond roughly to the settings on the Formats tab of the preference pane. One problem with trusting the LANG variable is that its UNIX semantics are as a fallback for the LC_* variables, not an override. Given that the user has selected a language and formats in System Preferences, that's like setting all of the various LC_* variables, so LANG should be ignored. Allowing LANG to override the user's preference settings would make Wine behave badly when run from Terminal.app if the user's preferred language and formats don't match (e.g. Japanese language, U.S. formats). Terminal (at least, recent versions) has a setting in its preferences to set the locale environment variables in shells it creates. However, it really only sets LANG according to the Region selected on the Formats tab. It does not set individual LC_* variables. It completely ignores the user's preferred language. If Wine were to rely on LANG, then all users with mismatched language and formats who launch it from Terminal would not get their preferred language. If you as a user want to override the language that Wine displays, you can set LC_MESSAGES in your environment rather than LANG. Setting LC_ALL would achieve that, plus override the various other locale categories. Set LANG is the wrong approach, in my opinion. -Ken
Ken Thomases wrote:
Setting LC_ALL would achieve that, plus override the various other locale categories. Set LANG is the wrong approach, in my opinion.
However this is not the way Wine works on all other platforms. Keeping everything in sync is more appropriate here. Or modify global Wine behavior. Vitaliy.
On Nov 26, 2009, at 12:31 AM, Vitaliy Margolen wrote:
Ken Thomases wrote:
Setting LC_ALL would achieve that, plus override the various other locale categories. Set LANG is the wrong approach, in my opinion.
However this is not the way Wine works on all other platforms.
Sure it is, if you consider the user preferences to be setting the LC_* variables. On Linux, if the LC_* variables are set, then setting LANG does nothing. LANG is, always and everywhere, a fallback, not an override. This is true for how Wine works on Linux and Mac OS X and everywhere else. I see no inconsistency. On Nov 26, 2009, at 12:58 AM, Dmitry Timoshkov wrote:
"Ken Thomases" <ken(a)codeweavers.com> wrote:
If you as a user want to override the language that Wine displays, you can set LC_MESSAGES in your environment rather than LANG. Setting LC_ALL would achieve that, plus override the various other locale categories. Set LANG is the wrong approach, in my opinion.
In the most cases what user wants is not just change the UI language (which setting the LC_MESSAGES does) but also change the locale (i.e. input/output code page), that's where LANG should be used.
No, that's where setting LC_ALL should be used.
LC_MESSAGES hack adds more confusion IMO than really helps.
Except that the vast, vast majority of Mac users will never touch (or even think about) the LANG or LC_* variables. For them, the current code does the right thing, even for the case where their preferred language is set to something other than their formats region. Letting LANG override their system settings would produce incorrect results for those users and force them to think about and use LANG or LC_ALL where they otherwise wouldn't. I see no upside at all to changing LANG into an override where that's not how it functions in any other context. Regards, Ken
"Ken Thomases" <ken(a)codeweavers.com> wrote:
If you as a user want to override the language that Wine displays, you can set LC_MESSAGES in your environment rather than LANG. Setting LC_ALL would achieve that, plus override the various other locale categories. Set LANG is the wrong approach, in my opinion.
In the most cases what user wants is not just change the UI language (which setting the LC_MESSAGES does) but also change the locale (i.e. input/output code page), that's where LANG should be used. LC_MESSAGES hack adds more confusion IMO than really helps. -- Dmitry.
Hi, As the author of the support for GNU gettext in MacOS X [1], I can explain the relation between the POSIX notion of locale and the MacOS X user preferences. POSIX:2008 [2] says: "All implementations shall define a locale as the default locale, to be invoked when no environment variables are set, or set to the empty string. This default locale can be the POSIX locale or any other implementation- defined locale. Some implementations may provide facilities for local installation administrators to set the default locale, customizing it for each location. POSIX.1-2008 does not require such a facility." This means that a POSIX compliant program has to do the following steps in order to determine the locale for a certain category of settings. 1) Inspect the LC_ALL environment variable. If it is set and non-empty, use its value. 2) Otherwise: Inspect the LC_xxxx environment variable that matches the specific category. If it is set and non-empty, use its value. 3) Otherwise: Inspect the LANG environment variable. If it is set and non-empty, use its value. 4) Otherwise: Use the default locale. On MacOS X systems, this means to use user preferences that can be retrieved using the CFLocaleCopyCurrent, CFPreferencesCopyAppValue functions. Note that these settings are similar but not entirely equal to Unix (glibc) conventions (e.g. "zh-Hans" vs. "zh_CN"), therefore some mapping of names has to be done. This algorithm is used in all programs that rely on GNU gettext and works perfectly fine: - By default, no LANG or LC_* environment variable is set, hence the programs obey the user settings. - For users of Terminal.app, who set these environment variables, the programs obey the environment variables. Ken Thomases wrote:
Mac OS X's Language & Text (formerly known as International) settings in System Preferences don't quite map exactly to Unix LANG or LC_* environment variables, which are a poor intermediary for mapping to Windows settings.
I agree that if you want to map from MacOS X locale names from CF* functions to Win32 locale names, you may want to do so without going through Unix (glibc) conventional names.
One problem with trusting the LANG variable is that its UNIX semantics are as a fallback for the LC_* variables, not an override. Given that the user has selected a language and formats in System Preferences, that's like setting all of the various LC_* variables, so LANG should be ignored.
No. If a program ignores LANG while it is set, it is not POSIX compliant, and a user will wonder why it behaves differently than all the POSIX compliant programs on his system.
Allowing LANG to override the user's preference settings would make Wine behave badly when run from Terminal.app if the user's preferred language and formats don't match (e.g. Japanese language, U.S. formats).
1. This is a rare case: Normally users run Terminal.app without setting LANG. 2. If they do set LANG, POSIX requires the program to obey LANG.
If Wine were to rely on LANG, then all users with mismatched language and formats who launch it from Terminal would not get their preferred language.
This would be compliant with POSIX, and with what GNU gettext does for years. No one ever complained about it.
If you as a user want to override the language that Wine displays, you can set LC_MESSAGES in your environment rather than LANG. Setting LC_ALL would achieve that, plus override the various other locale categories. Set LANG is the wrong approach, in my opinion.
According to POSIX, if a user sets LC_ALL=foo it is equivalent to unset LC_ALL; LC_CTYPE=foo LC_MESSAGES=foo LC_NUMERIC=foo LC_TIME=foo ... and equivalent to unset LC_ALL LC_CTYPE LC_MESSAGES LC_NUMERIC LC_TIME ...; LANG=foo Every program should treat these three ways to set the locale in the same way. If Wine was to behave differently in the second case than in the third case (by ignoring the LANG setting in the third case, as you propose), it would be buggy. Bruno [1] http://git.savannah.gnu.org/gitweb/?p=gettext.git;a=blob;f=gettext-runtime/i... [2] http://www.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap07.html
On Nov 30, 2009, at 5:31 PM, Bruno Haible wrote:
This means that a POSIX compliant program has to do the following steps in order to determine the locale for a certain category of settings. 1) Inspect the LC_ALL environment variable. If it is set and non-empty, use its value. 2) Otherwise: Inspect the LC_xxxx environment variable that matches the specific category. If it is set and non-empty, use its value. 3) Otherwise: Inspect the LANG environment variable. If it is set and non-empty, use its value. 4) Otherwise: Use the default locale. On MacOS X systems, this means to use user preferences that can be retrieved using the CFLocaleCopyCurrent, CFPreferencesCopyAppValue functions. Note that these settings are similar but not entirely equal to Unix (glibc) conventions (e.g. "zh-Hans" vs. "zh_CN"), therefore some mapping of names has to be done.
By this logic, virtual no end-user program on any given Mac is compliant. Gettext and POSIX-compliance is one thing. The user-desired behavior for Mac programs is another.
This algorithm is used in all programs that rely on GNU gettext and works perfectly fine: - By default, no LANG or LC_* environment variable is set, hence the programs obey the user settings. - For users of Terminal.app, who set these environment variables, the programs obey the environment variables.
It is Terminal.app itself which is setting LANG for all shells it creates, by default. The user is most likely oblivious to this fact.
Ken Thomases wrote:
Mac OS X's Language & Text (formerly known as International) settings in System Preferences don't quite map exactly to Unix LANG or LC_* environment variables, which are a poor intermediary for mapping to Windows settings.
I agree that if you want to map from MacOS X locale names from CF* functions to Win32 locale names, you may want to do so without going through Unix (glibc) conventional names.
It's not about mapping from "MacOS X locale names" (whatever you mean by that) that's the problem. It's the fact that no name or simple string properly represents the full range of settings available in System Preferences.
One problem with trusting the LANG variable is that its UNIX semantics are as a fallback for the LC_* variables, not an override. Given that the user has selected a language and formats in System Preferences, that's like setting all of the various LC_* variables, so LANG should be ignored.
No. If a program ignores LANG while it is set, it is not POSIX compliant, and a user will wonder why it behaves differently than all the POSIX compliant programs on his system.
I would venture to say that the number of Mac users who will wonder that is vanishingly small. Just about the only thing that will get a Mac user wondering is if the behavior of programs deviates from the settings in System Preferences. I would further venture that the number of POSIX compliant programs on the system with which the user has any familiarity is negligible. By your definition (which I don't question), no Mac applications are compliant.
Allowing LANG to override the user's preference settings would make Wine behave badly when run from Terminal.app if the user's preferred language and formats don't match (e.g. Japanese language, U.S. formats).
1. This is a rare case: Normally users run Terminal.app without setting LANG.
This is not a rare case. It's the default case. It will be nearly 100% of the time. Users do not set LANG; Terminal.app does!
2. If they do set LANG, POSIX requires the program to obey LANG.
And Wine is applying for POSIX certification? News to me.
If Wine were to rely on LANG, then all users with mismatched language and formats who launch it from Terminal would not get their preferred language.
This would be compliant with POSIX, and with what GNU gettext does for years. No one ever complained about it.
Very, very few Mac users use any program which uses gettext.
If you as a user want to override the language that Wine displays, you can set LC_MESSAGES in your environment rather than LANG. Setting LC_ALL would achieve that, plus override the various other locale categories. Set LANG is the wrong approach, in my opinion.
According to POSIX, if a user sets LC_ALL=foo it is equivalent to unset LC_ALL; LC_CTYPE=foo LC_MESSAGES=foo LC_NUMERIC=foo LC_TIME=foo ... and equivalent to unset LC_ALL LC_CTYPE LC_MESSAGES LC_NUMERIC LC_TIME ...; LANG=foo Every program should treat these three ways to set the locale in the same way. If Wine was to behave differently in the second case than in the third case (by ignoring the LANG setting in the third case, as you propose), it would be buggy.
Buggy in a sense which completely ignores user expectation. That's not the sense in which I'd use that word. The vast majority of users of Wine on the Mac will expect what I've proposed and be surprised and dismayed by what you've proposed. -Ken
Ken Thomases wrote:
On Nov 30, 2009, at 5:31 PM, Bruno Haible wrote:
This means that a POSIX compliant program has to do the following steps in order to determine the locale for a certain category of settings. 1) Inspect the LC_ALL environment variable. If it is set and non-empty, use its value. 2) Otherwise: Inspect the LC_xxxx environment variable that matches the specific category. If it is set and non-empty, use its value. 3) Otherwise: Inspect the LANG environment variable. If it is set and non-empty, use its value. 4) Otherwise: Use the default locale. On MacOS X systems, this means to use user preferences that can be retrieved using the CFLocaleCopyCurrent, CFPreferencesCopyAppValue functions. Note that these settings are similar but not entirely equal to Unix (glibc) conventions (e.g. "zh-Hans" vs. "zh_CN"), therefore some mapping of names has to be done.
By this logic, virtual no end-user program on any given Mac is compliant.
Gettext and POSIX-compliance is one thing. The user-desired behavior for Mac programs is another.
This algorithm is used in all programs that rely on GNU gettext and works perfectly fine: - By default, no LANG or LC_* environment variable is set, hence the programs obey the user settings. - For users of Terminal.app, who set these environment variables, the programs obey the environment variables.
It is Terminal.app itself which is setting LANG for all shells it creates, by default. The user is most likely oblivious to this fact.
The idea behind most of the MacIntosh 'magic' is that the user is deliberately oblivous to this. I just ran the set command in a terminal.app session, of which I have several running right now, and it was set to en_US.utf8, which is what I expect because the 'American' flag is displayed in the top information bar (also called the 'Apple' bar.) Gert may want to run the set command in a terminal session and given what was said, the LANG setting should be en_ZA.utf8, which is what I expect for South African English. The Wine project could use this as a default with the Mac, which is not POSIX compliant. James McKenzie
On Nov 30, 2009, at 9:34 PM, James McKenzie wrote:
This algorithm is used in all programs that rely on GNU gettext and works perfectly fine: - By default, no LANG or LC_* environment variable is set, hence the programs obey the user settings. - For users of Terminal.app, who set these environment variables, the programs obey the environment variables.
It is Terminal.app itself which is setting LANG for all shells it creates, by default. The user is most likely oblivious to this fact.
The idea behind most of the MacIntosh 'magic' is that the user is deliberately oblivous to this.
I agree that Wine (well, anything) should meet the user's expectations without the user having to even be aware of their expectations. Unfortunately, "respecting" LANG does not lead to this...
I just ran the set command in a terminal.app session, of which I have several running right now, and it was set to en_US.utf8, which is what I expect because the 'American' flag is displayed in the top information bar (also called the 'Apple' bar.) Gert may want to run the set command in a terminal session and given what was said, the LANG setting should be en_ZA.utf8, which is what I expect for South African English.
First, the language and formats have nothing to do with the keyboard layout (the U.S. flag in the input menu). But, more importantly for this discussion, the fact that LANG will be appropriate for you and probably Gert does not mean it will be appropriate "often enough" for Wine to rely on. As I have said, there are many ways in which users configure their language and formats settings that will not be properly reflected in LANG (and, indeed, cannot be).
The Wine project could use this as a default with the Mac, which is not POSIX compliant.
You are suggesting using a proxy for the real thing rather than the real thing. If we want to maintain the "magic", Wine should just use the settings from System Preferences directly (via CFLocale, CFNumberFormatter, and CFDateFormatter), with an optional way to override them for those rare users who want Wine's behavior to deviate from the system's. LANG doesn't help maintain the magic. It's a hindrance. Regards, Ken
On Tue, Dec 1, 2009 at 06:12, Ken Thomases <ken(a)codeweavers.com> wrote:
On Nov 30, 2009, at 9:34 PM, James McKenzie wrote:
The idea behind most of the MacIntosh 'magic' is that the user is deliberately oblivous to this.
I agree that Wine (well, anything) should meet the user's expectations without the user having to even be aware of their expectations. Unfortunately, "respecting" LANG does not lead to this...
An important question is: Which environment should Wine emulate under OS X? Unix apps under OS X, Windows in the same locale as OS X or OS X-native apps? (I tend to prefer emulating OS X by default and allowing overrides using LC_ALL (in order to allow one override method on all platforms, which LANG is unsuitable for, since it is a last resort option)) In addition Wine should leave the registry settings alone if the settings on which they were based were left unchanged. This allows a user to fully customize Wine's "regional settings" without changing anything in the environment where Wine is run.
I just ran the set command in a terminal.app session, of which I have several running right now, and it was set to en_US.utf8, which is what I expect because the 'American' flag is displayed in the top information bar (also called the 'Apple' bar.) Gert may want to run the set command in a terminal session and given what was said, the LANG setting should be en_ZA.utf8, which is what I expect for South African English.
I'm paste it in here soon... My Ubuntu settings is: mohag(a)mohagpc:~$ locale LANG=en_ZA.UTF-8 LANGUAGE= LC_CTYPE="en_ZA.UTF-8" LC_NUMERIC="en_ZA.UTF-8" LC_TIME="en_ZA.UTF-8" LC_COLLATE="en_ZA.UTF-8" LC_MONETARY="en_ZA.UTF-8" LC_MESSAGES="en_ZA.UTF-8" LC_PAPER="en_ZA.UTF-8" LC_NAME="en_ZA.UTF-8" LC_ADDRESS="en_ZA.UTF-8" LC_TELEPHONE="en_ZA.UTF-8" LC_MEASUREMENT="en_ZA.UTF-8" LC_IDENTIFICATION="en_ZA.UTF-8" LC_ALL= My OS X settings is: (Language set to "English", "Formats" set to "South Africa") MoHaGLaptop:~ mohag$ locale LANG= LC_COLLATE="C" LC_CTYPE="UTF-8" LC_MESSAGES="C" LC_MONETARY="C" LC_NUMERIC="C" LC_TIME="C" LC_ALL= I would interpret that as default language, C for numbers, currency, etc, which would not be correct. (I think a (Perl/Python if any of them support native locale under all OSes?) script to quickly test the behaviour under Wine / OS X / *nix might be handy to figure out what applications are actually seeing....)
But, more importantly for this discussion, the fact that LANG will be appropriate for you and probably Gert does not mean it will be appropriate "often enough" for Wine to rely on. As I have said, there are many ways in which users configure their language and formats settings that will not be properly reflected in LANG (and, indeed, cannot be).
The Wine project could use this as a default with the Mac, which is not POSIX compliant.
You are suggesting using a proxy for the real thing rather than the real thing. If we want to maintain the "magic", Wine should just use the settings from System Preferences directly (via CFLocale, CFNumberFormatter, and CFDateFormatter), with an optional way to override them for those rare users who want Wine's behavior to deviate from the system's.
That makes sense to me... (If Wine tries to be a good OS X app under OS X, rather than a good POSIX app under OS X) (And what is suggested was current Wine settings of environment remained the same, LC_ALL (to allow for similar override everywhere), platform specific settings (e.g. individual settings from OS X's "Formats" settings), LC_* (which only contains specific settings, not "country" / "language"), LANG (for unset settings, and locale/country itself). On OS X this should mean that the variables other than LC_ALL only gets looked at if reading the OS X settings fail for some reason) For consistency with *nix override via LC_ALL makes sense (ignoring all other LC_*/LANG settings on OS X, in favour of OS X specific settings)
LANG doesn't help maintain the magic. It's a hindrance.
LC_ALL should be used to override settings, not LANG. LANG doesn't work (on Linux at least) if LC_ALL is defined. Telling users to use LANG if it only works in certain cases does not make sense... mohag(a)mohagpc:~$ LC_ALL="en_US.UTF-8" LANG="en_ZA.UTF-8" wine regedit /E - 'HKEY_CURRENT_USER\Control Panel\International'|egrep "Country|Locale" "iCountry"="1" "Locale"="00000409" "sCountry"="United States" mohag(a)mohagpc:~$ LC_ALL="en_ZA.UTF-8" LANG="en_US.UTF-8" wine regedit /E - 'HKEY_CURRENT_USER\Control Panel\International'|egrep "Country|Locale" "iCountry"="27" "Locale"="00001c09" "sCountry"="South Africa" I'm actually quite impressed with Wine's handling of the locale under Linux, it seem to be possible to overwrite the individual settings with LC_*, which I did not think was possible based on Vitality's comments on Wine-users. In addition it seems not to mess around with the settings if the environment stays the same. (Which is mostly the behaviour that I prefer, since it starts with defaults based on my settings (whether it is based on Windows' or Linux's idea of my settings is not easy to test, since I don't know of any obvious differences to test) and allows me to customize it under Wine to fix broken defaults / specific settings needed by applications) Under Linux it seems to emulate native apps quite nicely. I don't currently have an OS X Wine installed to test effect of different settings. The environment variables does seem to be affected by the language settings, which might be caused by OS X's decoupling of "Formats" settings and "Language" settings, which can result in some weird combinations. (e.g. Dutch language with "Formats" country set to "China" ) (Parts of this might be in need of some reordering, had internet issues and might have repeated myself) Gert
On Dec 1, 2009, at 1:30 PM, Gert van den Berg wrote:
My OS X settings is: (Language set to "English", "Formats" set to "South Africa") MoHaGLaptop:~ mohag$ locale LANG= LC_COLLATE="C" LC_CTYPE="UTF-8" LC_MESSAGES="C" LC_MONETARY="C" LC_NUMERIC="C" LC_TIME="C" LC_ALL=
Are you using Terminal.app or some other terminal application? If Terminal.app, have you turned off the "Set LANG environment variable on startup" setting in preferences? (That's the name on Leopard. If I recall, that setting is slightly different in Snow Leopard: "Set locale environment variables on startup" or something like that.)
On Tue, Dec 1, 2009 at 06:12, Ken Thomases <ken(a)codeweavers.com> wrote:
[...] Wine should just use the settings from System Preferences directly (via CFLocale, CFNumberFormatter, and CFDateFormatter), with an optional way to override them for those rare users who want Wine's behavior to deviate from the system's.
That makes sense to me... (If Wine tries to be a good OS X app under OS X, rather than a good POSIX app under OS X)
(And what is suggested was current Wine settings of environment remained the same, LC_ALL (to allow for similar override everywhere), platform specific settings (e.g. individual settings from OS X's "Formats" settings), LC_* (which only contains specific settings, not "country" / "language"), LANG (for unset settings, and locale/country itself). On OS X this should mean that the variables other than LC_ALL only gets looked at if reading the OS X settings fail for some reason)
For consistency with *nix override via LC_ALL makes sense (ignoring all other LC_*/LANG settings on OS X, in favour of OS X specific settings)
Actually, I think that Wine should respect the LC_* variables if they are set. So, the precedence order would be LC_ALL > LC_* > Mac OS X System Preferences > LANG. If one of the LC_* variables are set, that expresses user intent (because those variables are not set by Terminal.app automatically). Also, it is useful to allow the user to override the System Preferences on a category-by-category basis. Cheers, Ken
On Wed, Dec 2, 2009 at 18:05, Ken Thomases <ken(a)codeweavers.com> wrote:
Are you using Terminal.app or some other terminal application? If Terminal.app, have you turned off the "Set LANG environment variable on startup" setting in preferences? (That's the name on Leopard. If I recall, that setting is slightly different in Snow Leopard: "Set locale environment variables on startup" or something like that.)
I'm using Terminal.app using default settings (AFAIK, except for colors). "Set locale variables at startup" is enabled. I don't see settings for en_ZA* in /usr/share/locale (af_ZA does exist, but if my applications turn Afrikaans I'll go insane ;-) ) (And OS X does not have a language settign for Afrikaans)
Actually, I think that Wine should respect the LC_* variables if they are set. So, the precedence order would be LC_ALL > LC_* > Mac OS X System Preferences > LANG.
As far as I can figure out, no LC_* variable, except LC_ALL specific the actual locale? They only determine individual settings, which overwrites the default for the locale, as set by LANG? (Not sure if it effectively means set everything to LANG's locale, overwrite individual settings from LC_* if they are defined)
If one of the LC_* variables are set, that expresses user intent (because those variables are not set by Terminal.app automatically). Also, it is useful to allow the user to override the System Preferences on a category-by-category basis.
Mine gets set to C / UTF-8(for LC_CTYPE) of some reason... Gert
On Dec 2, 2009, at 1:22 PM, Gert van den Berg wrote:
I'm using Terminal.app using default settings (AFAIK, except for colors). "Set locale variables at startup" is enabled.
I don't see settings for en_ZA* in /usr/share/locale (af_ZA does exist, but if my applications turn Afrikaans I'll go insane ;-) ) (And OS X does not have a language settign for Afrikaans)
Apparently Terminal.app won't set LANG to a locale that's not in /usr/ share/locale.
Actually, I think that Wine should respect the LC_* variables if they are set. So, the precedence order would be LC_ALL > LC_* > Mac OS X System Preferences > LANG.
As far as I can figure out, no LC_* variable, except LC_ALL specific the actual locale? They only determine individual settings, which overwrites the default for the locale, as set by LANG? (Not sure if it effectively means set everything to LANG's locale, overwrite individual settings from LC_* if they are defined)
I'm not sure what you mean here. Are you referring to the general Unix meaning of these variables or Wine's use of them? As near as I can tell, Wine only probes the locale information by invoking setlocale(<category>, NULL). It doesn't directly examine the environment variables. Furthermore, it only queries using the individual categories, not LC_ALL. It uses setlocale(LC_ALL, "") once to tell the C library to initialize itself from the environment. So, I'm not sure how setting all of the LC_* variables to a given value can be different from setting LC_ALL to that value.
If one of the LC_* variables are set, that expresses user intent (because those variables are not set by Terminal.app automatically). Also, it is useful to allow the user to override the System Preferences on a category-by-category basis.
Mine gets set to C / UTF-8(for LC_CTYPE) of some reason...
Maybe Terminal.app is setting LC_CTYPE when it can't set LANG. Check what's actually set in your environment: env | egrep "^(LANG|LC_)" Regards, Ken
On Wed, Dec 2, 2009 at 22:52, Ken Thomases <ken(a)codeweavers.com> wrote:
On Dec 2, 2009, at 1:22 PM, Gert van den Berg wrote:
As far as I can figure out, no LC_* variable, except LC_ALL specific the actual locale? They only determine individual settings, which overwrites the default for the locale, as set by LANG? (Not sure if it effectively means set everything to LANG's locale, overwrite individual settings from LC_* if they are defined)
I'm not sure what you mean here. Are you referring to the general Unix meaning of these variables or Wine's use of them?
General *nix meaning. What I meant is that LC_* (not LC_ALL) don't have a setting that says "the locale is say en_ZA" it only has we have number (LC_NUMERIC) from en_ZA, times (LC_TIME) from en_UK, sort things (LC_COLLATE) according to en_AU, determine whether something is a letter / number and how to change their case (LC_CTYPE) according to UTF-8, ect. LANG is only setting that really maps to Wine / Windows' sCountry and Locale settings. (if all LC_* variables are set, LANG only determine things not in any of the LC_* variables) LC_* does not have anything to determine the actual UI language either. What I meant above, is that you should be able to get the equivalent effect (to the POSIX default) by setting everything to LANG's setting and then changing them from the LC_* variables. E.g. if LANG is fr_FR (not sure if that is an actual name) and LC_TIME is en_US, set environment up for French, change settings affected by LC_TIME to US English. Setting LANG to French and all individual settings to en_US, shoulld reslut in a system where the country is France, Locale code is 0x040c, UI is French, but date / time / number / currency settings, etc. is that of en_US. Considering how OS X seem to set variables, that might work as well... My interpretation of your proposed logic for OS X: Set up everything to OS X setting, overwrite everything if LANG is set, overwrite further settings from LC_*, overwrite everything from LC_ALL if that is set. (the slow unoptimized way that might result in some settings being overwritten 4 times before settling on a value...) Resulting order: LC_ALL LC_* LANG OS X settings.
As near as I can tell, Wine only probes the locale information by invoking setlocale(<category>, NULL). It doesn't directly examine the environment variables. Furthermore, it only queries using the individual categories, not LC_ALL. It uses setlocale(LC_ALL, "") once to tell the C library to initialize itself from the environment.
So, I'm not sure how setting all of the LC_* variables to a given value can be different from setting LC_ALL to that value.
Interpreting it as above, settings LC_ALL is equivalent to setting all LC_ variables and setting LANG to LC_ALL's value.
If one of the LC_* variables are set, that expresses user intent (because those variables are not set by Terminal.app automatically). Also, it is useful to allow the user to override the System Preferences on a category-by-category basis.
Mine gets set to C / UTF-8(for LC_CTYPE) of some reason...
Maybe Terminal.app is setting LC_CTYPE when it can't set LANG. Check what's actually set in your environment:
env | egrep "^(LANG|LC_)"
Sorry, only LC_CTYPE is actually set, to UTF-8. (form the locale command's output one would think that it does the same, which it obviously doesn't) Looking at the current values, your order for OS X makes sense. If the actual effect is similar to the result of the overwriting above, you can overwrite settings as in *nix, but OS X's ones are used by default. (Not sure what it will do if OS X actually knows your locale?) Gert
On Dec 2, 2009, at 11:25 PM, Gert van den Berg wrote:
What I meant is that LC_* (not LC_ALL) don't have a setting that says "the locale is say en_ZA" it only has we have number (LC_NUMERIC) from en_ZA, times (LC_TIME) from en_UK, sort things (LC_COLLATE) according to en_AU, determine whether something is a letter / number and how to change their case (LC_CTYPE) according to UTF-8, ect. LANG is only setting that really maps to Wine / Windows' sCountry and Locale settings. (if all LC_* variables are set, LANG only determine things not in any of the LC_* variables) LC_* does not have anything to determine the actual UI language either.
What I meant above, is that you should be able to get the equivalent effect (to the POSIX default) by setting everything to LANG's setting and then changing them from the LC_* variables. E.g. if LANG is fr_FR (not sure if that is an actual name) and LC_TIME is en_US, set environment up for French, change settings affected by LC_TIME to US English. Setting LANG to French and all individual settings to en_US, shoulld reslut in a system where the country is France, Locale code is 0x040c, UI is French, but date / time / number / currency settings, etc. is that of en_US.
I don't think that setting LC_ALL (or LANG) has any meaning beyond what one gets from the individual LC_* variables, but I could be mistaken. I understand that you're suggesting it should or that you can imagine a semantic meaning to setting all of the LC_* variables to one thing and LANG to another, but I don't think it actually works that way. In other words, I don't believe the C library has any notion of "the locale is en_ZA" in the sense you mean. It only has the various categories. Currently, Wine uses the LC_MESSAGES category to determine sCountry and Locale.
Considering how OS X seem to set variables, that might work as well... My interpretation of your proposed logic for OS X: Set up everything to OS X setting, overwrite everything if LANG is set, overwrite further settings from LC_*, overwrite everything from LC_ALL if that is set. (the slow unoptimized way that might result in some settings being overwritten 4 times before settling on a value...)
Resulting order: LC_ALL LC_* LANG OS X settings.
That's not quite what I proposed. I propose the following (in pseudocode): mac_formats_locale = CFLocaleGetIdentifier( CFLocaleCopyCurrent() ); mac_language = CFArrayGetValueAtIndex( CFBundleCopyLocalizationsForPreferences( CFLocaleCopyAvailableLocaleIdentifiers(), NULL ), 0 ); for cat in ( LC_COLLATE, LC_CTYPE, LC_MONETARY, LC_NUMERIC, LC_TIME, LC_PAPER, LC_MEASUREMENT, LC_TELEPHONE ) setenv( cat, mac_formats_locale, 0 ); setenv( "LC_MESSAGES", mac_language, 0 ); The 0 as the third argument to setenv() means don't replace any existing definition. In this way, any LC_* environment variables set in the original environment would take precedence over the System Preferences settings. (LC_PAPER, LC_MEASUREMENT, and LC_TELEPHONE would only be used if actually defined, which they currently aren't on Mac OS X.) The above would all precede the call to setlocale( LC_ALL, "" ) so that it would influence how the C library locale gets initialized. We would remove the post-setlocale override of lcid_LC_MESSAGES. This results in the order I previously described: LC_ALL LC_* from the original environment Mac OS X settings LANG The Mac OS X settings have to take precedence over LANG for the reasons I've previously described. (Terminal.app sets it automatically to a value that doesn't/can't quite represent the user's actual preferences. As it happens, Terminal.app isn't doing this for you because Mac OS X doesn't support en_ZA in the Unix locale definitions in /usr/share/locale.)
As near as I can tell, Wine only probes the locale information by invoking setlocale(<category>, NULL). It doesn't directly examine the environment variables. Furthermore, it only queries using the individual categories, not LC_ALL. It uses setlocale(LC_ALL, "") once to tell the C library to initialize itself from the environment.
So, I'm not sure how setting all of the LC_* variables to a given value can be different from setting LC_ALL to that value.
Interpreting it as above, settings LC_ALL is equivalent to setting all LC_ variables and setting LANG to LC_ALL's value.
No. LANG is irrelevant if all LC_* variables are set. It's only use is to supply the locale for a category if LC_ALL or the specific LC_whatever is not set. If you set all of the LC_whatevers, then LANG is unused. Regards, Ken
On Dec 3, 2009, at 11:27 AM, Ken Thomases wrote:
I propose the following (in pseudocode):
mac_formats_locale = CFLocaleGetIdentifier( CFLocaleCopyCurrent() ); mac_language = CFArrayGetValueAtIndex ( CFBundleCopyLocalizationsForPreferences ( CFLocaleCopyAvailableLocaleIdentifiers(), NULL ), 0 ); for cat in ( LC_COLLATE, LC_CTYPE, LC_MONETARY, LC_NUMERIC, LC_TIME, LC_PAPER, LC_MEASUREMENT, LC_TELEPHONE ) setenv( cat, mac_formats_locale, 0 ); setenv( "LC_MESSAGES", mac_language, 0 );
I have encountered a problem with my proposal. The above pseudo-code for determining mac_language is quite likely to produce a string like "en" or "zh_Hans" -- that is, a locale specifier with no country code. Setting LC_MESSAGES to a locale string without a country code will cause the C library to consider it invalid. (That's not a requirement explicitly imposed by the C library, but a consequence of what's in /usr/share/locale. For example, the command "locale -a" does not list any locales without country codes.) When that happens, it considers all LC_* variables invalid and defaults to the "C" locale. The code that Wine currently uses avoids this problem because it doesn't go through the C library. Basically, after all of the LANG/ setlocale/setup_unix_locales stuff has been done, it overwrites the value in lcid_LC_MESSAGES (a Wine variable). So, I'm making a new proposal: the Mac formats region will be used to set LANG unconditionally. (Passing 1 for the third argument to setenv() rather than 0.) The current code for overwriting lcid_LC_MESSAGES will be tweaked. Instead of looking at whether lcid_LC_MESSAGES has been set to something other than lcid_LC_CTYPE, it will look at whether LC_ALL or LC_MESSAGES were set in the environment. If they were, it leaves lcid_LC_MESSAGES alone. If they were not, it overwrites it using the existing method. This retains the precedence order we desire: LC_ALL LC_* Mac OS X settings LANG because: * LANG is just replaced with the Mac OS X region * If LC_ALL or the LC_* variables are set, they take precedence over LANG in the C library (setlocale) * If LC_ALL or LC_MESSAGES are set, we don't overwrite lcid_LC_MESSAGES after the C library has done its thing Does that sound sensible? -Ken
Ken Thomases wrote:
On Dec 3, 2009, at 11:27 AM, Ken Thomases wrote:
I propose the following (in pseudocode):
mac_formats_locale = CFLocaleGetIdentifier( CFLocaleCopyCurrent() ); mac_language = CFArrayGetValueAtIndex( CFBundleCopyLocalizationsForPreferences( CFLocaleCopyAvailableLocaleIdentifiers(), NULL ), 0 ); for cat in ( LC_COLLATE, LC_CTYPE, LC_MONETARY, LC_NUMERIC, LC_TIME, LC_PAPER, LC_MEASUREMENT, LC_TELEPHONE ) setenv( cat, mac_formats_locale, 0 ); setenv( "LC_MESSAGES", mac_language, 0 );
I have encountered a problem with my proposal.
The above pseudo-code for determining mac_language is quite likely to produce a string like "en" or "zh_Hans" -- that is, a locale specifier with no country code. Setting LC_MESSAGES to a locale string without a country code will cause the C library to consider it invalid. (That's not a requirement explicitly imposed by the C library, but a consequence of what's in /usr/share/locale. For example, the command "locale -a" does not list any locales without country codes.) When that happens, it considers all LC_* variables invalid and defaults to the "C" locale.
The code that Wine currently uses avoids this problem because it doesn't go through the C library. Basically, after all of the LANG/setlocale/setup_unix_locales stuff has been done, it overwrites the value in lcid_LC_MESSAGES (a Wine variable).
So, I'm making a new proposal: the Mac formats region will be used to set LANG unconditionally. (Passing 1 for the third argument to setenv() rather than 0.) The current code for overwriting lcid_LC_MESSAGES will be tweaked. Instead of looking at whether lcid_LC_MESSAGES has been set to something other than lcid_LC_CTYPE, it will look at whether LC_ALL or LC_MESSAGES were set in the environment. If they were, it leaves lcid_LC_MESSAGES alone. If they were not, it overwrites it using the existing method.
This retains the precedence order we desire:
LC_ALL LC_* Mac OS X settings LANG
because:
* LANG is just replaced with the Mac OS X region * If LC_ALL or the LC_* variables are set, they take precedence over LANG in the C library (setlocale) * If LC_ALL or LC_MESSAGES are set, we don't overwrite lcid_LC_MESSAGES after the C library has done its thing
Does that sound sensible?
-Ken
LGTM Chip
On Dec 9, 2009, at 1:34 PM, Ken Thomases wrote:
So, I'm making a new proposal: the Mac formats region will be used to set LANG unconditionally. (Passing 1 for the third argument to setenv() rather than 0.) The current code for overwriting lcid_LC_MESSAGES will be tweaked. Instead of looking at whether lcid_LC_MESSAGES has been set to something other than lcid_LC_CTYPE, it will look at whether LC_ALL or LC_MESSAGES were set in the environment. If they were, it leaves lcid_LC_MESSAGES alone. If they were not, it overwrites it using the existing method.
Now I've reconsidered a bit more. I'm not going to overwrite LANG unconditionally. For most of this discussion, I've been pretty hostile toward LANG. The main problem with it is that Terminal.app sets it, this setting is based on region and not language, and therefore if we honor LANG for LC_MESSAGES, the user gets the wrong language. Given that we're going to continue overwriting lcid_LC_MESSAGES using the Mac OS X settings, just under different conditions than previously, Terminal's LANG is not so problematic. In fact, it's likely exactly equal to what our code will now generate for LANG. (Gert's reporting shows that Terminal sometimes doesn't set LANG at all and only sets LC_CTYPE, because it detects that the region doesn't have a valid locale definition for the C library. In this case, Wine will set LANG, because it isn't already set, but that will be invalid and everything will default to the C locale, including LC_CTYPE. This actually loses information. Unfortunately, there's not much we can do about that. Wine has to set LANG if it isn't set because, for example, Tiger's Terminal.app does not make any attempt to set locale environment variables.) The big change between now and the beginning of this debate is that we must acknowledge that the way to tell Wine to behave differently (e.g. to test other languages and locales) is to set LC_ALL or the individual LC_* variables. I still maintain that setting LANG has never been the right approach for that. And setting LANG will continue to be an incomplete override on the Mac (it won't get LC_MESSAGES), as it is on Linux for a user with an LC_* in his or her everyday environment. -Ken
participants (7)
-
Bruno Haible -
Charles Davis -
Dmitry Timoshkov -
Gert van den Berg -
James McKenzie -
Ken Thomases -
Vitaliy Margolen