Following a discussion with Alexandre on IRC, here is a revised patch. This one doesn't touch the LC_MESSAGE, as these do not affect the user locale. Further work will be necessary on supporting MUI like language checking.
Shachar Shemesh wrote:
Changelog: Shachar Shemesh winecode@shemesh.biz dlls/kernel/locale.c
- System default locale in Windows determines the ANSI encoding (LC_CTYPE on Unix)
"Shachar Shemesh" wine-patches@shemesh.biz wrote:
Following a discussion with Alexandre on IRC, here is a revised patch. This one doesn't touch the LC_MESSAGE, as these do not affect the user locale. Further work will be necessary on supporting MUI like language checking.
if ((lang = getenv( "LC_ALL" )) ||
(lang = getenv( "LANGUAGE" )) ||
(env1 && (lang = getenv( env1 ))) || (lang = getenv( "LANG" )))
I think that re-introducing an ability to change Wine's locale with LC_CTYPE is flawed. Remeber, that current ANSI code page affects not only keyboard input, but also all things depending on MultiByteToWideChar/WideCharToMultiByte conversions, that's all resource strings and hardcoded non-unicode application strings.
If you are trying to fix keyboard input, you have to fix it not in Wine, but in your Linux locale setup. If you have LC_ALL, LANG and LC_TYPE pointing to different locales your setup is broken.
Dmitry Timoshkov wrote:
"Shachar Shemesh" wine-patches@shemesh.biz wrote:
Following a discussion with Alexandre on IRC, here is a revised patch. This one doesn't touch the LC_MESSAGE, as these do not affect the user locale. Further work will be necessary on supporting MUI like language checking.
if ((lang = getenv( "LC_ALL" )) ||
(lang = getenv( "LANGUAGE" )) ||
(env1 && (lang = getenv( env1 ))) || (lang = getenv( "LANG" )))I think that re-introducing an ability to change Wine's locale with LC_CTYPE is flawed.
That's not what I'm trying to do.
Remeber, that current ANSI code page affects not only keyboard input, but also all things depending on MultiByteToWideChar/WideCharToMultiByte conversions, that's all resource strings and hardcoded non-unicode application strings.
Good. That's the effect I'm trying to achieve here. LC_CTYPE controls what codepage the system is in. Wine should inherit that.
If you are trying to fix keyboard input, you have to fix it not in Wine, but in your Linux locale setup. If you have LC_ALL, LANG and LC_TYPE pointing to different locales your setup is broken.
If I have LC_ALL pointing to something, then that's what everything is. LANG, on the other hand, is defined to the lowest precedence there is, AFTER LC_*.
If I have LC_CTYPE and LANG pointing at different things, it may be because I want to speak one language, but have an encoding that belongs to another. There is nothing broken with that, and there is no reason not to support that.
Shachar
"Shachar Shemesh" wine-devel@shemesh.biz wrote:
If I have LC_CTYPE and LANG pointing at different things, it may be because I want to speak one language, but have an encoding that belongs to another. There is nothing broken with that, and there is no reason not to support that.
That can't work, since there is only one active locale in the win32 world. Your problems are caused by X11 limitations, that X11 has no a notion of input locale, has no distinct locale for an active keyboard layout, and uses current system locale instead. They adopted UTF-8 as a workaround for the whole problem instead of fixing it properly.
Dmitry Timoshkov wrote:
"Shachar Shemesh" wine-devel@shemesh.biz wrote:
If I have LC_CTYPE and LANG pointing at different things, it may be because I want to speak one language, but have an encoding that belongs to another. There is nothing broken with that, and there is no reason not to support that.
That can't work, since there is only one active locale in the win32 world.
Huh? With my patch LC_CTYPE affects the "system default locale". LC_ALL, LANGUAGE (a mistake, I think, but couldn't convince Alexandre), and LANG affect the "user locale". Hence - there are two locales in the Windows world. If we implement the MUI interfaces, we will actually have three.
Your problems are caused by X11 limitations, that X11 has no a notion of input locale, has no distinct locale for an active keyboard layout, and uses current system locale instead.
You are referring to problems I'm having with detecting keyboard languages that have nothing to do with this patch. This patch is not an attempt to solve those problems, and has nothing to do with them. It also should not affect them.
They adopted UTF-8 as a workaround for the whole problem instead of fixing it properly.
Shachar
"Shachar Shemesh" wine-devel@shemesh.biz wrote:
With my patch LC_CTYPE affects the "system default locale". LC_ALL, LANGUAGE (a mistake, I think, but couldn't convince Alexandre), and LANG affect the "user locale". Hence - there are two locales in the Windows world. If we implement the MUI interfaces, we will actually have three.
No. GetACP() returns exactly one value, initialized at the boot time, and CP_ACP pseudo code page used by multibyte<->unicode conversions is that only one active locale I mentioned above. You can't solve your problem by just using terms "system" and "user" locale, that changes nothing.
Dmitry Timoshkov wrote:
"Shachar Shemesh" wine-devel@shemesh.biz wrote:
With my patch LC_CTYPE affects the "system default locale". LC_ALL, LANGUAGE (a mistake, I think, but couldn't convince Alexandre), and LANG affect the "user locale". Hence - there are two locales in the Windows world. If we implement the MUI interfaces, we will actually have three.
No. GetACP() returns exactly one value, initialized at the boot time, and CP_ACP pseudo code page used by multibyte<->unicode conversions is that only one active locale I mentioned above. You can't solve your problem by just using terms "system" and "user" locale, that changes nothing.
Frankly, I'm not sure what problem that is. Dmitry, please explain what you find wrong with this patch.
Windows has a property called "system locale". This property affects what codepage is used for non-unicode applications.
Unix has an almost identical property, which selects how to interpret a string of bytes. This is selected by LC_ALL if it's defined, if not then by LC_CTYPE if it's defined, and if that's not defined then by LANG (man 7 locale).
My patch initializes system locale according to LC_ALL if it's defined, LC_CTYPE if not, and LANG if both are not defined.
Can you please explain what you see as wrong with that?
Shachar
"Shachar Shemesh" wine-devel@shemesh.biz wrote:
Frankly, I'm not sure what problem that is. Dmitry, please explain what you find wrong with this patch.
Windows has a property called "system locale". This property affects what codepage is used for non-unicode applications.
Even more confusingly there is also thread locale, but still it has nothing to do with things you are trying to fix.
Unix has an almost identical property, which selects how to interpret a string of bytes. This is selected by LC_ALL if it's defined, if not then by LC_CTYPE if it's defined, and if that's not defined then by LANG (man 7 locale).
My patch initializes system locale according to LC_ALL if it's defined, LC_CTYPE if not, and LANG if both are not defined.
Can you please explain what you see as wrong with that?
As far as I understand what you are trying to do and what the patch Alexandre committed does is to make the user interface use english while you still have an ability to type in your native language.
That's not how Windows does this, at least to my understanding. On my english win2k with locale set to russian both GetSystemDefaultLCID and GetUserDefaultLCID return 0419 (russian). I don't know how MUI enabled Windows versions implement that functionality, but I don't think that it's just the system locale.
Dmitry Timoshkov wrote:
As far as I understand what you are trying to do and what the patch Alexandre committed does is to make the user interface use english while you still have an ability to type in your native language.
No. What I'm trying to do is have none-unicode applications use the correct codepage according to their LC_CTYPE. That's all I'm trying to do. While my original patch tried to also choose the interface language, I removed that part. I think you are simply reading into this patch's intentions things that are not there.
That's not how Windows does this, at least to my understanding.
That is precisely how windows does it.
On my english win2k with locale set to russian both GetSystemDefaultLCID and GetUserDefaultLCID return 0419 (russian).
Now I'm not following what you are trying to say. According to my understanding of Windows, if you installed an English Win2k and set system locale to Russian, all non-unicode applications you run will interpret the string of bytes they read from files etc. under a Russian codepage. If you set the user locale to Russian as well, you will get dates in European order etc. as well. Your interface language should remain English. That is the behaviour you are seeing, right? In fact, the only way you would actually get Russian interface is if you installed a Russian Windows. If you then set your system locale to English, you would get question marks instead of letters from many non-unicode applications that use resources, as their resources would try to load Russian, which would not be supported by their codepage (system locale).
Your setup is equivalent to setting "LANG" to Russian, and LC_MESSAGES to en_US on Unix, or to setting LANG to en_US and then setting LC_CTYPE, LC_CDATE etc. to Russian. Either one should do the trick on Unix, and so either one should do the trick on Wine.
I don't know how MUI enabled Windows versions implement that functionality, but I don't think that it's just the system locale.
It's not. I've installed MUI, and am currently investigating this. I'm thinking of either having "FindResource" use the interface language as defined by MUI instead of "LANG_NEUTRAL, SUBLANG_NEUTRAL", or replace entry 9 in "find_entry" with the above mentioned language. I still have to see how that is defined on Windows MUI. In any case, I trust there is no argument that Wine should behave like MUI, right?
Either way, the patch above (second version) had no presumptions to do what you claim it tries to do, and I therefor think your objection to it is unfounded.
Shachar
"Shachar Shemesh" wine-devel@shemesh.biz wrote:
As far as I understand what you are trying to do and what the patch Alexandre committed does is to make the user interface use english while you still have an ability to type in your native language.
No. What I'm trying to do is have none-unicode applications use the correct codepage according to their LC_CTYPE. That's all I'm trying to do. While my original patch tried to also choose the interface language, I removed that part. I think you are simply reading into this patch's intentions things that are not there.
Why LC_ALL or LANG don't work for you then?
That's not how Windows does this, at least to my understanding.
That is precisely how windows does it.
On my english win2k with locale set to russian both GetSystemDefaultLCID and GetUserDefaultLCID return 0419 (russian).
Now I'm not following what you are trying to say. According to my understanding of Windows, if you installed an English Win2k and set system locale to Russian, all non-unicode applications you run will interpret the string of bytes they read from files etc. under a Russian codepage.
No, data from any sources doesn't get interpreted by apps, the strings are interpreted when they are passed to A family of APIs.
If you set the user locale to Russian as well, you will get dates in European order etc. as well. Your interface language should remain English. That is the behaviour you are seeing, right?
You are confusing two different things: language of the user interface (you call it the system locale) and current ANSI code page (you call it the user locale). There only one locale of the system which defines current ANSI code page, language of the user interface has nothing to do with it.
I don't know how MUI enabled Windows versions implement that functionality, but I don't think that it's just the system locale.
It's not. I've installed MUI, and am currently investigating this. I'm thinking of either having "FindResource" use the interface language as defined by MUI instead of "LANG_NEUTRAL, SUBLANG_NEUTRAL", or replace entry 9 in "find_entry" with the above mentioned language. I still have to see how that is defined on Windows MUI. In any case, I trust there is no argument that Wine should behave like MUI, right?
Right, it just should be implemented properly, and not like an one liner hack.
Either way, the patch above (second version) had no presumptions to do what you claim it tries to do, and I therefor think your objection to it is unfounded.
My objection is that your patch aiming a good intention does the things in the wrong way.
Dmitry Timoshkov wrote:
"Shachar Shemesh" wine-devel@shemesh.biz wrote:
As far as I understand what you are trying to do and what the patch Alexandre committed does is to make the user interface use english while you still have an ability to type in your native language.
No. What I'm trying to do is have none-unicode applications use the correct codepage according to their LC_CTYPE. That's all I'm trying to do. While my original patch tried to also choose the interface language, I removed that part. I think you are simply reading into this patch's intentions things that are not there.
Why LC_ALL or LANG don't work for you then?
Please reread the patch. They are looked at, just not exclusively. LC_ALL overrides everything, and LANG is looked at if no LC_CTYPE (or whatever) is used. That's the order Unix does it (man 7 locale).
On my english win2k with locale set to russian both GetSystemDefaultLCID and GetUserDefaultLCID return 0419 (russian).
Now I'm not following what you are trying to say. According to my understanding of Windows, if you installed an English Win2k and set system locale to Russian, all non-unicode applications you run will interpret the string of bytes they read from files etc. under a Russian codepage.
No, data from any sources doesn't get interpreted by apps, the strings are interpreted when they are passed to A family of APIs.
Same difference. Still, this interpretation is determined by the system locale on Windows, and by LC_CTYPE on Unix. Hence - LC_CTYPE should affect system locale.
If you set the user locale to Russian as well, you will get dates in European order etc. as well. Your interface language should remain English. That is the behaviour you are seeing, right?
You are confusing two different things: language of the user interface (you call it the system locale)
No. Language of interface, which I call "the language of windows you installed".
and current ANSI code page (you call it the user locale).
No. I call that the system locale.
There only one locale of the system which defines current ANSI code page,
Which is the system locale, taken (according to my patch) from LC_ALL if defined, LC_CTYPE if not defined, and LANG if neither are defined. Why do you object to that?
language of the user interface has nothing to do with it.
Nor was it meant to be affected by this patch (the second version - the one that was committed). Nor, coming to it, is it actually affected.
It's not. I've installed MUI, and am currently investigating this. I'm thinking of either having "FindResource" use the interface language as defined by MUI instead of "LANG_NEUTRAL, SUBLANG_NEUTRAL", or replace entry 9 in "find_entry" with the above mentioned language. I still have to see how that is defined on Windows MUI. In any case, I trust there is no argument that Wine should behave like MUI, right?
Right, it just should be implemented properly, and not like an one liner hack.
again - this one liner is not attempting it.
Either way, the patch above (second version) had no presumptions to do what you claim it tries to do, and I therefor think your objection to it is unfounded.
My objection is that your patch aiming a good intention does the things in the wrong way.
I'll repeat - you misinterpret the patch's intention. You therefor come to the conclusion that it is a hack. It is neither aimed at affecting the user interface language, nor does it in actuality. Please reread the second version of the patch.
Shachar
"Shachar Shemesh" wine-devel@shemesh.biz wrote:
Why LC_ALL or LANG don't work for you then?
Please reread the patch. They are looked at, just not exclusively. LC_ALL overrides everything, and LANG is looked at if no LC_CTYPE (or whatever) is used. That's the order Unix does it (man 7 locale).
Well, I had read glibc sources and it's definitely ignores LC_CTYPE.
No, data from any sources doesn't get interpreted by apps, the strings are interpreted when they are passed to A family of APIs.
Same difference. Still, this interpretation is determined by the system locale on Windows, and by LC_CTYPE on Unix. Hence - LC_CTYPE should affect system locale.
No, it shouldn't. Again, I'd ask to use LANG or LC_ALL for that purpose, you can't have LANG and LC_TYPE point to different locales in a sane manner anyway.
Sorry, I skipped other parts of your arguments.
Dmitry Timoshkov wrote:
"Shachar Shemesh" wine-devel@shemesh.biz wrote:
Why LC_ALL or LANG don't work for you then?
Please reread the patch. They are looked at, just not exclusively. LC_ALL overrides everything, and LANG is looked at if no LC_CTYPE (or whatever) is used. That's the order Unix does it (man 7 locale).
Well, I had read glibc sources and it's definitely ignores LC_CTYPE.
Maybe there is no part of glibc that cares what encoding it is. I can tell you with authority that X does look at it, as does less, konsole, kedit, and almost any other application I have ever run.
No, it shouldn't. Again, I'd ask to use LANG or LC_ALL for that purpose, you can't have LANG and LC_TYPE point to different locales in a sane manner anyway.
? Docs says I can. Actual practice says I can (I do it all the time, as do many people in Israel). Please elaborate on why you chose to call a documented behavior *that works* "insane".
Sorry, I skipped other parts of your arguments.
Shachar
"Shachar Shemesh" wine-devel@shemesh.biz wrote:
Well, I had read glibc sources and it's definitely ignores LC_CTYPE.
Maybe there is no part of glibc that cares what encoding it is. I can tell you with authority that X does look at it, as does less, konsole, kedit, and almost any other application I have ever run.
Sure they do, because they handle keyboard input on their own.
No, it shouldn't. Again, I'd ask to use LANG or LC_ALL for that purpose, you can't have LANG and LC_TYPE point to different locales in a sane manner anyway.
? Docs says I can. Actual practice says I can (I do it all the time, as do many people in Israel). Please elaborate on why you chose to call a documented behavior *that works* "insane".
I explained it many times already: because you can't have different locales for keyboard input, string collation and to/from unicode conversion routines.
Dmitry Timoshkov wrote:
? Docs says I can. Actual practice says I can (I do it all the time, as do many people in Israel). Please elaborate on why you chose to call a documented behavior *that works* "insane".
I explained it many times already: because you can't have different locales for keyboard input, string collation and to/from unicode conversion routines.
Ok, it seems that we are walking around in circles here.
Windows has it. Just set system locale to one thing (code page, to/from unicode), and user locale to something else (collation). Unix has it (LC_COLLATION and LC_CTYPE). Why shouldn't wine?
Even if you are right that it leads to insanity (and I do understand why you say that, at least for the codepage and collation collision. If there are any other cases of insanity, do tell), aren't we supposed to be bug-compatible with Windows?
Shachar
"Shachar Shemesh" wine-devel@shemesh.biz wrote:
I explained it many times already: because you can't have different locales for keyboard input, string collation and to/from unicode conversion routines.
Ok, it seems that we are walking around in circles here.
Windows has it. Just set system locale to one thing (code page, to/from unicode), and user locale to something else (collation). Unix has it (LC_COLLATION and LC_CTYPE). Why shouldn't wine?
I still didn't hear from you from you are calling user locale in Windows, that's the source of misunderstanding I guess.
Even if you are right that it leads to insanity (and I do understand why you say that, at least for the codepage and collation collision. If there are any other cases of insanity, do tell), aren't we supposed to be bug-compatible with Windows?
No, if there is no applications depending on it.
Dmitry Timoshkov wrote:
"Shachar Shemesh" wine-devel@shemesh.biz wrote:
I explained it many times already: because you can't have different locales for keyboard input, string collation and to/from unicode conversion routines.
Ok, it seems that we are walking around in circles here.
Windows has it. Just set system locale to one thing (code page, to/from unicode), and user locale to something else (collation). Unix has it (LC_COLLATION and LC_CTYPE). Why shouldn't wine?
I still didn't hear from you from you are calling user locale in Windows, that's the source of misunderstanding I guess.
User locale determines collation and default date, time and monetary formats. More precisely, it determines the default thread locale, which then determines all of the above.
If you read up the thread, you will see an email by me asking what should be good values to initialize user locale from. The patch as committed uses only LC_ALL and LANG, due to not finding a good answer to the above question. I'm slowly coming to the conclusion that LC_COLLATE would be a good answer, if we handle LC_DATE etc. properly (i.e. - override the default if it's different). However, I did not do enough research to reach a definite answer on this one.
Also, as I mentioned before, I've installed MUI, and will look into how %(!*#@@!& it determines it's interface language. Once we have an answer to that, we can finally also honor LC_MESSAGES (and LANGUAGE), and get a truly locale aware Wine.
Even if you are right that it leads to insanity (and I do understand why you say that, at least for the codepage and collation collision. If there are any other cases of insanity, do tell), aren't we supposed to be bug-compatible with Windows?
No, if there is no applications depending on it.
But if there hadn't been any application depending on it, I wouldn't bother, would I? Considering the amount of applications written for Unicode (almost nil), it is safe to say that almost EVERY application is affected by it, if your environment is set up like that. Like I said before, having LANG point at one thing, and LC_CTYPE at another is a pretty common setup in Israel.
You can call it wrong, and I would welcome a crusade to fix the common Israeli setup. I just don't think rejecting this patch will do anything to help the process. Even if I accept your logic, this patch is fully GIGO compliant while not breaking anything if your setup is "right".
Shachar
Shachar Shemesh wine-devel@shemesh.biz writes:
With my patch LC_CTYPE affects the "system default locale". LC_ALL, LANGUAGE (a mistake, I think, but couldn't convince Alexandre), and LANG affect the "user locale".
Actually I agree that LANGUAGE is a mistake. What your original patch did was making the mistake even worse by adding LC_MESSAGES to the mix, and that is what I objected too. I'm perfectly happy with not taking LANGUAGE into account for the user locale, it should only be used for the UI language, just like LC_MESSAGES.
Alexandre Julliard wrote:
Shachar Shemesh wine-devel@shemesh.biz writes:
With my patch LC_CTYPE affects the "system default locale". LC_ALL, LANGUAGE (a mistake, I think, but couldn't convince Alexandre), and LANG affect the "user locale".
Actually I agree that LANGUAGE is a mistake. What your original patch did was making the mistake even worse by adding LC_MESSAGES to the mix, and that is what I objected too. I'm perfectly happy with not taking LANGUAGE into account for the user locale, it should only be used for the UI language, just like LC_MESSAGES.
The reason my original patch was written the way it was is *because* "LANGUAGE" was there. I'm in the process of installing Windows MUI, so I can write a test case for how to change the UI language.
Do we actually query the user locale anywhere? The docs (http://support.microsoft.com/default.aspx?scid=kb;en-us;258047) suggests that it is only used as a default for such things as the date format, time format, etc. However, on Unix these are controlled by separate environment variables, and even on Windows they are overrideable on a per-setting basis (through the regional settings control panel). It seems like the user locale is one of the most useless settings available, with everybody attributing the wrong functionality to it.
So it would appear that the correct thing to do is to create a list of mapping. In the following list, LC_ALL always overrides the list member, and LANG is always taken if the neither the list member nor LC_ALL are set. We already have: LC_CTYPE -> system locale none -> user locale (take LC_ALL, and if not set, LANG) LC_COLLATE -> no setting (user-locale?) LC_MONETARY -> Specific registry entries LC_MESSAGES & LANGUAGE -> only settable on MUI versions of Windows LC_NUMERIC -> Specific registry entries? LC_MONETARY -> Specific registry entries LC_PAPER -> Per printer setting LC_MEASUREMENT -> Specific registry entries LC_TIME -> specific registry entries.
I also have no idea what LC_IDENTIFICATION, LC_NAME or LC_ADDRESS mean.
Whenever I wrote "Specific registry entries" in the above table, I meant that it's taken from the user locale unless it's overridden by the registry. The best thing for us to do, I think, is to set the user locale according to LC_ALL, LC_COLLATE, LANG, and then query each of the other strings and set the registry according to them.
Shachar
-
Alexandre Julliard -
Dmitry Timoshkov -
Dmitry Timoshkov -
Shachar Shemesh -
Shachar Shemesh