Re: Locales with the same language

[Dmitry Timoshkov]

"Marcelo Duarte"  wrote:
> If I understanded, people that have LANG=fr_CA or pt_PT, etc (non-default 
> SUBLANG), will have to use English?

If there are no resources with SUBLANGE_DEFAULT, then yes.
> Then the correct will be change all resources with SUBLANGE_DEFAULT to 
> SUBLANG_NEUTRAL until someone localize them?
> I can modify this if is the correct.

Actually there was a completely opposite patch committed recently:

2003-01-05
    Shachar Shemesh 
    Change the SUBLANG_NEUTRAL clause in all winelib applications to
    SUBLANG_DEFAULT, as they should be.

I vote to revert that patch.

Locales with the same language

[Shachar Shemesh]

Dmitry Timoshkov wrote:
> > Then the correct will be change all resources with SUBLANGE_DEFAULT to 
> > SUBLANG_NEUTRAL until someone localize them?
> > I can modify this if is the correct.
>
> Actually there was a completely opposite patch committed recently:
>
> 2003-01-05
>    Shachar Shemesh 
>    Change the SUBLANG_NEUTRAL clause in all winelib applications to
>    SUBLANG_DEFAULT, as they should be.
>
> I vote to revert that patch.

Then perhaps I should explain why I submitted this patch.

The problem was with notepad. If my locale said "he_IL", it would 
display the menues in Danish. The reason was that, failing to find 
Hebrew locales, and failing to find LANG_ENGLISH, SUBLANG_DEFAULT 
locales, it loaded the first one available.

I then proceeded to research what the situation on Windows was, and the 
categorical answer was that Visual Studio would not let you create 
resources with SUBLANG_NEUTRAL. SUBLANG_DEFAULT was the only option. It 
is for that reason that I believe that SUBLANG_NEUTRAL should only be 
used as a search qualifier, never as an actual resource.

Now, looking at the code searching path I see:
    /* 2. specified language with neutral sublanguage */
    pos = push_language( list, pos, MAKELANGID( 
PRIMARYLANGID(info->Language), SUBLANG_NEUTRAL ) );

As far as my understanding goes, this should match the LANG_FRENCH with 
ANY sublang. Why doesn't this rule correctly find LANG_FRENCH,
SUBLANG_FRENCH_CANADIAN? The MSDN is a little hazy on the subject of 
SUBLANG_NEUTRAL. It has always been my understanding that we should be 
treating SUBLANG_NEUTRAL as a wildcard. Is that wrong?

Shachar

Locales with the same language

[Dmitry Timoshkov]

"Shachar Shemesh"  wrote:
> > 2003-01-05
> >    Shachar Shemesh 
> >    Change the SUBLANG_NEUTRAL clause in all winelib applications to
> >    SUBLANG_DEFAULT, as they should be.
> >
> > I vote to revert that patch.
>
> Then perhaps I should explain why I submitted this patch.
>
> The problem was with notepad. If my locale said "he_IL", it would 
> display the menues in Danish. The reason was that, failing to find 
> Hebrew locales, and failing to find LANG_ENGLISH, SUBLANG_DEFAULT 
> locales, it loaded the first one available.

Then probably you had to replace SUBLANG_NEUTRAL by SUBLANG_DEFAULT
only for LANG_ENGLISH resources. Anyway the patch I just set adds
LANG_ENGLISH,SUBLANG_NEUTRAL to the fallback list, since my test
under win2k shows that.
> I then proceeded to research what the situation on Windows was, and the 
> categorical answer was that Visual Studio would not let you create 
> resources with SUBLANG_NEUTRAL. SUBLANG_DEFAULT was the only option. It 
> is for that reason that I believe that SUBLANG_NEUTRAL should only be 
> used as a search qualifier, never as an actual resource.

If VS wouldn't allow you to add Hebrew resources you would decide that
Hebrew language is forbidden in resources?
> Now, looking at the code searching path I see:
>     /* 2. specified language with neutral sublanguage */
>     pos = push_language( list, pos, MAKELANGID( 
> PRIMARYLANGID(info->Language), SUBLANG_NEUTRAL ) );
>
> As far as my understanding goes, this should match the LANG_FRENCH with 
> ANY sublang.

No, it adds match for LANG_FRENCH,SUBLANG_NEUTRAL *only*.
> Why doesn't this rule correctly find LANG_FRENCH,
> SUBLANG_FRENCH_CANADIAN?

Because there is no resources marked for that language dialect.
> The MSDN is a little hazy on the subject of 
> SUBLANG_NEUTRAL. It has always been my understanding that we should be 
> treating SUBLANG_NEUTRAL as a wildcard. Is that wrong?

The best way is to write a test program. I did it. Did you?

Locales with the same language

[Shachar Shemesh]

Dmitry Timoshkov wrote:
> "Shachar Shemesh"  wrote:
>
> Then probably you had to replace SUBLANG_NEUTRAL by SUBLANG_DEFAULT
> only for LANG_ENGLISH resources. Anyway the patch I just set adds
> LANG_ENGLISH,SUBLANG_NEUTRAL to the fallback list, since my test
> under win2k shows that.

After some more analysis as a result of our discussion, maybe that is, 
indeed, the right thing to do.
> > I then proceeded to research what the situation on Windows was, and the 
> > categorical answer was that Visual Studio would not let you create 
> > resources with SUBLANG_NEUTRAL. SUBLANG_DEFAULT was the only option. It 
> > is for that reason that I believe that SUBLANG_NEUTRAL should only be 
> > used as a search qualifier, never as an actual resource.
>
> If VS wouldn't allow you to add Hebrew resources you would decide that
> Hebrew language is forbidden in resources?

VS DOESN'T let me add Hebrew resources, but if I then go and type in the 
language ID manually, it accepts it. On the other hand, it didn't accept 
SUBLANG_NEUTRAL as a valid option. As I said earlier in this mail, 
further research seems to suggest that I was, indeed, wrong.
> > The MSDN is a little hazy on the subject of 
> > SUBLANG_NEUTRAL. It has always been my understanding that we should be 
> > treating SUBLANG_NEUTRAL as a wildcard. Is that wrong?
>
> The best way is to write a test program. I did it. Did you?

At the time, yes. The result was that LANG_ENGLISH, SUBLANG_DEFAULT was 
selected no matter what I did. When I have the time (yeah, right) I'll 
try installing a few variants of the same windows versions and do 
comparative assesment. I'm willing to mark this down as a bad case of 
memory on my part, in any case.

Shachar

Locales with the same language

[Dmitry Timoshkov]

"Shachar Shemesh"  wrote:
> > Then probably you had to replace SUBLANG_NEUTRAL by SUBLANG_DEFAULT
> > only for LANG_ENGLISH resources. Anyway the patch I just set adds
> > LANG_ENGLISH,SUBLANG_NEUTRAL to the fallback list, since my test
> > under win2k shows that.
>
> After some more analysis as a result of our discussion, maybe that is, 
> indeed, the right thing to do.

Could you ask Alexandre to revert your patch please?

Locales with the same language

[Vincent Béron]

Le jeu 17/07/2003 à 11:18, Dmitry Timoshkov a écrit :
> "Shachar Shemesh"  wrote:
> > > Then probably you had to replace SUBLANG_NEUTRAL by SUBLANG_DEFAULT
> > > only for LANG_ENGLISH resources. Anyway the patch I just set adds
> > > LANG_ENGLISH,SUBLANG_NEUTRAL to the fallback list, since my test
> > > under win2k shows that.
> >
> > After some more analysis as a result of our discussion, maybe that is, 
> > indeed, the right thing to do.
>
> Could you ask Alexandre to revert your patch please?

Using SUBLANG_NEUTRAL in notepad rather than SUBLANG_DEFAULT makes it
speak French on both Windows and Wine with LANG=fr_CA, LANG=fr_CH, etc.

Vincent

Locales with the same language

[Vincent Béron]

Le jeu 17/07/2003 à 12:05, Vincent Béron a écrit :
> Le jeu 17/07/2003 à 11:18, Dmitry Timoshkov a écrit :
> > "Shachar Shemesh"  wrote:
> > > > Then probably you had to replace SUBLANG_NEUTRAL by SUBLANG_DEFAULT
> > > > only for LANG_ENGLISH resources. Anyway the patch I just set adds
> > > > LANG_ENGLISH,SUBLANG_NEUTRAL to the fallback list, since my test
> > > > under win2k shows that.
> > >
> > > After some more analysis as a result of our discussion, maybe that is, 
> > > indeed, the right thing to do.
> >
> > Could you ask Alexandre to revert your patch please?
>
> Using SUBLANG_NEUTRAL in notepad rather than SUBLANG_DEFAULT makes it
> speak French on both Windows and Wine with LANG=fr_CA, LANG=fr_CH, etc.

(after a bit more testing)
Using only SUBLANG_FRENCH_CANADIAN will display in French on both
Windows and Wine.
Using only SUBLANG_FRENCH_SWISS will display in English on both Windows
and Wine.
Using only SUBLANG_FRENCH will display in French only in Windows,
English in Wine (with LANG=fr_CA).

So SUBLANG_FRENCH seems to act (on Windows) the same way as a wildcard,
matching any of it's sublanguages if an exact match can't be done.

I also tried defining more than one (ie, SUBLANG_FRENCH and
SUBLANG_FRENCH_CANADIAN) resources. On Windows it chose the
SUBLANG_FRENCH, while in Wine it chose the SUBLANG_FRENCH_CANADIAN. That
gets me a bit puzzled though... unless my installation of Windows is set
to French, then French Canadian as a fallback (don't know of to check
this).

Vincent

Locales with the same language

[Dmitry Timoshkov]

"Vincent Béron"  wrote:
> (after a bit more testing)
> Using only SUBLANG_FRENCH_CANADIAN will display in French on both
> Windows and Wine.
> Using only SUBLANG_FRENCH_SWISS will display in English on both Windows
> and Wine.
> Using only SUBLANG_FRENCH will display in French only in Windows,
> English in Wine (with LANG=fr_CA).
>
> So SUBLANG_FRENCH seems to act (on Windows) the same way as a wildcard,
> matching any of it's sublanguages if an exact match can't be done.

Could you please elaborate a bit, how exactly you did the tests above?
> I also tried defining more than one (ie, SUBLANG_FRENCH and
> SUBLANG_FRENCH_CANADIAN) resources. On Windows it chose the
> SUBLANG_FRENCH, while in Wine it chose the SUBLANG_FRENCH_CANADIAN. That
> gets me a bit puzzled though... unless my installation of Windows is set
> to French, then French Canadian as a fallback (don't know of to check
> this).

All of this highly depend on the exact language set in your system (be it
Windows or Wine).

I've attached my test app. See the results I got eclosed there between
#if 0/#endif.

Note, that if your system doesn't support SetThreadLocale() (i.e. you are
running Win9x), the system will prefer your current locale settings
instead of the provided value and it will influence behaviour of LoadString
and FindResource.

As you might see, Wine doesn't behave exactly like Windows, Wine makes
a preference for the current locale, Windows prefers to choose LANG_ENGLISH
first. As it was discussed previously, Wine's behaviour have to stay as
it is now, otherwise all NLS support for built-in DLLs and apps will broke.

How I did my test:
1. Uncomment all resource strings in locales.rc. If you want, you could
add your own resource strings. Make N = 1.
2. build the test.
3. Run test.exe 0x40C
(0x40C == LANG_FRENCH,SUBLANG_DEFAULT). Note that SUBLANG_FRENCH == SUBLANG_DEFAULT).
4. Write down the result under number N. N = N + 1.
5. Comment out the found resource.
6. Goto 2.

--
Dmitry.

Locales with the same language

[Shachar Shemesh]

Dmitry Timoshkov wrote:
> "Vincent Béron"  wrote:
> > (after a bit more testing)
> > Using only SUBLANG_FRENCH_CANADIAN will display in French on both
> > Windows and Wine.
> > Using only SUBLANG_FRENCH_SWISS will display in English on both Windows
> > and Wine.
> > Using only SUBLANG_FRENCH will display in French only in Windows,
> > English in Wine (with LANG=fr_CA).
> >
> > So SUBLANG_FRENCH seems to act (on Windows) the same way as a wildcard,
> > matching any of it's sublanguages if an exact match can't be done.
>
> Could you please elaborate a bit, how exactly you did the tests above?
> > I also tried defining more than one (ie, SUBLANG_FRENCH and
> > SUBLANG_FRENCH_CANADIAN) resources. On Windows it chose the
> > SUBLANG_FRENCH, while in Wine it chose the SUBLANG_FRENCH_CANADIAN. That
> > gets me a bit puzzled though... unless my installation of Windows is set
> > to French, then French Canadian as a fallback (don't know of to check
> > this).
>
> All of this highly depend on the exact language set in your system (be it
> Windows or Wine).
>
> I've attached my test app. See the results I got eclosed there between
> #if 0/#endif.
>
> Note, that if your system doesn't support SetThreadLocale() (i.e. you are
> running Win9x), the system will prefer your current locale settings
> instead of the provided value and it will influence behaviour of LoadString
> and FindResource.
>
> As you might see, Wine doesn't behave exactly like Windows, Wine makes
> a preference for the current locale, Windows prefers to choose LANG_ENGLISH
> first. As it was discussed previously, Wine's behaviour have to stay as
> it is now, otherwise all NLS support for built-in DLLs and apps will broke.

I suspect that this depends on the language of Windows you install. When 
I have some time, I'll play a bit with this. MS also has a version of 
windows called "MUI", or Multilinugual User Interface. I suspect that 
that version behaves more like Wine does at the moment, but I have not 
checked that one either.

In the mean while, let's indeed reverese the patch translating 
SUBLANG_NEUTRAL->SUBLANG_DEFAULT.

Shachar

Locales with the same language

[Vincent Béron]

Le ven 18/07/2003 à 09:32, Shachar Shemesh a écrit :
> Dmitry Timoshkov wrote:
> > "Vincent Béron"  wrote:
> > > (after a bit more testing)
> > > Using only SUBLANG_FRENCH_CANADIAN will display in French on both
> > > Windows and Wine.
> > > Using only SUBLANG_FRENCH_SWISS will display in English on both Windows
> > > and Wine.
> > > Using only SUBLANG_FRENCH will display in French only in Windows,
> > > English in Wine (with LANG=fr_CA).
> > >
> > > So SUBLANG_FRENCH seems to act (on Windows) the same way as a wildcard,
> > > matching any of it's sublanguages if an exact match can't be done.
> >
> > Could you please elaborate a bit, how exactly you did the tests above?

Compiling Wine's notepad in VC6 with different resource files: one for
English, and a couple for LANG_FRENCH/SUBLANG_NEUTRAL, SUBLANG_FRENCH,
SUBLANG_FRENCH_CANADIAN and SUBLANG_FRENCH_SWISS. Actually, that was for
the second batch of results (below).
The ones above come from changing the LANG_FRENCH/SUBLANG_DEFAULT of
pristine notepad to SUBLANG_FRENCH_CANADIAN and SUBLANG_FRENCH_SWISS.
> > > I also tried defining more than one (ie, SUBLANG_FRENCH and
> > > SUBLANG_FRENCH_CANADIAN) resources. On Windows it chose the
> > > SUBLANG_FRENCH, while in Wine it chose the SUBLANG_FRENCH_CANADIAN. That
> > > gets me a bit puzzled though... unless my installation of Windows is set
> > > to French, then French Canadian as a fallback (don't know of to check
> > > this).
> >
> > All of this highly depend on the exact language set in your system (be it
> > Windows or Wine).
> >
> > I've attached my test app. See the results I got eclosed there between
> > #if 0/#endif.
> >
> > Note, that if your system doesn't support SetThreadLocale() (i.e. you are
> > running Win9x), the system will prefer your current locale settings
> > instead of the provided value and it will influence behaviour of LoadString
> > and FindResource.

W2k here, so this is valid only for it.
LANGIDFROMLCID(GetLocaleThread()) returns 0x0c0c
(FRENCH/FRENCH_CANADIAN) if I don't modify it via SetThreadLocale(), so
I guess that's the default value for that system (equivalent to
LC_ALL=fr_CA).

The results I got (for LoadString and FindResource) are different from
yours though (still W2k). Here's for test.exe 0x0c0c:
1. LANG_NEUTRAL/SUBLANG_NEUTRAL
2. LANG_FRENCH/SUBLANG_FRENCH (==SUBLANG_DEFAULT)
3. LANG_FRENCH/SUBLANG_NEUTRAL
4. LANG_FRENCH/SUBLANG_FRENCH_CANADIAN
5. LANG_ENGLISH/SUBLANG_DEFAULT
6. LANG_ARABIC/SUBLANG_NEUTRAL

So our #2, #3 and #5 are different. Are they dependant on the system
default language? I believe it's more logical to look for the same
language as the current locale before a harcoded English, else as soon
as there's an English resource (DEFAULT or NEUTRAL) nothing else will be
displayed.

Here are the results for test.exe 0x0419 (RUSSIAN/DEFAULT) on the same
system:
1. NEUTRAL/NEUTRAL
2. RUSSIAN/DEFAULT
3. FRENCH/DEFAULT
4. FRENCH/NEUTRAL
5. FRENCH/FRENCH_CANADIAN
6. ENGLISH/DEFAULT
7. ARABIC/NEUTRAL

So we see it goes for current language id/default before system
default/default, and current language id/neutral is not looked for at
all.

Dmitry, on which language have you tested your program under Windows?
> > As you might see, Wine doesn't behave exactly like Windows, Wine makes
> > a preference for the current locale, Windows prefers to choose LANG_ENGLISH
> > first. As it was discussed previously, Wine's behaviour have to stay as
> > it is now, otherwise all NLS support for built-in DLLs and apps will broke.
>
> I suspect that this depends on the language of Windows you install. When 
> I have some time, I'll play a bit with this. MS also has a version of 
> windows called "MUI", or Multilinugual User Interface. I suspect that 
> that version behaves more like Wine does at the moment, but I have not 
> checked that one either.
>
> In the mean while, let's indeed reverese the patch translating 
> SUBLANG_NEUTRAL->SUBLANG_DEFAULT.

Wine's notepad uses LoadString, so it'll look for SUBLANG_DEFAULT before
SUBLANG_NEUTRAL. Hence I support the reversing of the patch.

Vincent

Locales with the same language

[Dmitry Timoshkov]

"Vincent Béron"  wrote:
> W2k here, so this is valid only for it.
> LANGIDFROMLCID(GetLocaleThread()) returns 0x0c0c
> (FRENCH/FRENCH_CANADIAN) if I don't modify it via SetThreadLocale(), so
> I guess that's the default value for that system (equivalent to
> LC_ALL=fr_CA).
>
> The results I got (for LoadString and FindResource) are different from
> yours though (still W2k). Here's for test.exe 0x0c0c:
> - LANG_NEUTRAL/SUBLANG_NEUTRAL
> - LANG_FRENCH/SUBLANG_FRENCH (==SUBLANG_DEFAULT)
> - LANG_FRENCH/SUBLANG_NEUTRAL
> - LANG_FRENCH/SUBLANG_FRENCH_CANADIAN
> - LANG_ENGLISH/SUBLANG_DEFAULT
> - LANG_ARABIC/SUBLANG_NEUTRAL
>
> So our #2, #3 and #5 are different. Are they dependant on the system
> default language? I believe it's more logical to look for the same
> language as the current locale before a harcoded English, else as soon
> as there's an English resource (DEFAULT or NEUTRAL) nothing else will be
> displayed.
>
> Here are the results for test.exe 0x0419 (RUSSIAN/DEFAULT) on the same
> system:
> - NEUTRAL/NEUTRAL
> - RUSSIAN/DEFAULT
> - FRENCH/DEFAULT
> - FRENCH/NEUTRAL
> - FRENCH/FRENCH_CANADIAN
> - ENGLISH/DEFAULT
> - ARABIC/NEUTRAL
>
> So we see it goes for current language id/default before system
> default/default, and current language id/neutral is not looked for at
> all.
>
> Dmitry, on which language have you tested your program under Windows?

I'm running English version of win2k pro with regional settings set to Russia.

Here are the results for 0x0c0c and 0x0419 on my system.

test.exe 0x0c0c
1. LANG_NEUTRAL/SUBLANG_NEUTRAL
2. LANG_FRENCH/SUBLANG_FRENCH_CANADIAN
3. LANG_ENGLISH/SUBLANG_DEFAULT
4. LANG_ENGLISH/SUBLANG_NEUTRAL
5. LANG_RUSSIAN/SUBLANG_DEFAULT
6. LANG_RUSSIAN/SUBLANG_NEUTRAL
7. LANG_ARABIC/SUBLANG_NEUTRAL

test.exe 0x0419
1. LANG_NEUTRAL/SUBLANG_NEUTRAL
2. LANG_ENGLISH/SUBLANG_DEFAULT
3. LANG_ENGLISH/SUBLANG_NEUTRAL
4. LANG_RUSSIAN/SUBLANG_DEFAULT
5. LANG_RUSSIAN/SUBLANG_NEUTRAL
6. LANG_ARABIC/SUBLANG_NEUTRAL

It seems that we have correct order of defaults in dlls/ntdll/resource.c
and my patch adding LANG_ENGLISH/SUBLANG_NEUTRAL is wrong. It would be
correct to add current thread locale language with neutral sublanguage
after #4 though.

--
Dmitry.