kernel32 nls file codepage conversion

[Frédéric Delanoy]

Hi folks,

I'm intending to convert dlls/kernel32/nls/fr.nls from CP 1252 to CP
65001 (UTF-8), but I've some questions first:
- most nls files are non-UTF8... is it intended? or just "legacy"?
- can LOCALE_IDEFAULTANSICODEPAGE be set to 65001 or should it be left
at its current "1252" value, i.e. is it necessary for compatibility
with older win versions where UTF-8 is not the default (correct me if
I'm wrong)?
- LOCALE_SCURRENCY is set to char code 80 (<80> in vim). What is that
exactly? The code for the euro sign in a specific  code page? which
one?

Frédéric

[Dmitry Timoshkov]

Frédéric Delanoy  wrote:
> I'm intending to convert dlls/kernel32/nls/fr.nls from CP 1252 to CP
> 65001 (UTF-8),

Is there a ny reason behind that?
> but I've some questions first:
> - most nls files are non-UTF8... is it intended? or just "legacy"?

nls files in Wine use Windows code page appropriate for the language
in that file, which a resource compiler understands, and which can be
corerctly converted to unicode.
> - can LOCALE_IDEFAULTANSICODEPAGE be set to 65001 or should it be left
>
> at its current "1252" value, i.e. is it necessary for compatibility
> with older win versions where UTF-8 is not the default (correct me if
> I'm wrong)?

Noone Windows version uses UTF-8 for locales. LOCALE_IDEFAULTANSICODEPAGE
must be set to the ANSI code page used by Windows for that laguage (i.e.
leave it alone).
> - LOCALE_SCURRENCY is set to char code 80 (<80> in vim). What is that
>
> exactly? The code for the euro sign in a specific  code page? which
> one?

The code page the file uses (1252 for fr.nls).

[Frédéric Delanoy]

2010/5/30 Dmitry Timoshkov :
> Frédéric Delanoy  wrote:
> > I'm intending to convert dlls/kernel32/nls/fr.nls from CP 1252 to CP
> > 65001 (UTF-8),
>
> Is there a ny reason behind that?

Uniformisation? Universality? All other rc files (at least Fr ones,
and most of the others) have already been
converted.
> > but I've some questions first:
> > - most nls files are non-UTF8... is it intended? or just "legacy"?
>
> nls files in Wine use Windows code page appropriate for the language
> in that file, which a resource compiler understands, and which can be
> corerctly converted to unicode.

So it can be in any code page, provided the #pragma codepage is set
accordingly I guess?
> > - can LOCALE_IDEFAULTANSICODEPAGE be set to 65001 or should it be left
> >
> > at its current "1252" value, i.e. is it necessary for compatibility
> > with older win versions where UTF-8 is not the default (correct me if
> > I'm wrong)?
>
> Noone Windows version uses UTF-8 for locales. LOCALE_IDEFAULTANSICODEPAGE
> must be set to the ANSI code page used by Windows for that laguage (i.e.
> leave it alone).

OK. If I understand correctly, the codepage of the nls file is
orthogonal to that of LOCALE_IDEFAULTANSICODEPAGE?
> > - LOCALE_SCURRENCY is set to char code 80 (<80> in vim). What is that
> >
> > exactly? The code for the euro sign in a specific  code page? which
> > one?
>
> The code page the file uses (1252 for fr.nls).

OK
> --
> Dmitry.

Thanks for your comments.

Frédéric

[Michael Stefaniuc]

On 05/30/2010 06:05 PM, Frédéric Delanoy wrote:
> 2010/5/30 Dmitry Timoshkov:
> > Frédéric Delanoy  wrote:
> > > I'm intending to convert dlls/kernel32/nls/fr.nls from CP 1252 to CP
> > > 65001 (UTF-8),
> >
> > Is there a ny reason behind that?
>
> Uniformisation? Universality? All other rc files (at least Fr ones,
> and most of the others) have already been
> converted.

Yeah, go ahead. There are already a few nls file in UTF-8. It would be 
good to have only one codepage used throughout the Wine code and UTF-8 
is the way to go. Alexandre isn't opposed to this but the translators 
have to do/request it.
> > > but I've some questions first:
> > > - most nls files are non-UTF8... is it intended? or just "legacy"?
> >
> > nls files in Wine use Windows code page appropriate for the language
> > in that file, which a resource compiler understands, and which can be
> > corerctly converted to unicode.
>
> So it can be in any code page, provided the #pragma codepage is set
> accordingly I guess?

Well, all the codepages that provide the needed characters ;)
> > > - can LOCALE_IDEFAULTANSICODEPAGE be set to 65001 or should it be left
> > >
> > > at its current "1252" value, i.e. is it necessary for compatibility
> > > with older win versions where UTF-8 is not the default (correct me if
> > > I'm wrong)?
> >
> > Noone Windows version uses UTF-8 for locales. LOCALE_IDEFAULTANSICODEPAGE
> > must be set to the ANSI code page used by Windows for that laguage (i.e.
> > leave it alone).
>
> OK. If I understand correctly, the codepage of the nls file is
> orthogonal to that of LOCALE_IDEFAULTANSICODEPAGE?

Yes.
> > > - LOCALE_SCURRENCY is set to char code 80 (<80>  in vim). What is that
> > >
> > > exactly? The code for the euro sign in a specific  code page? which
> > > one?
> >
> > The code page the file uses (1252 for fr.nls).
>
> OK

Yeah, most editors and afair file too misinterpret CP1252 as ISO-8859-1; 
they are very similar but not the same. Using UTF-8 "fixes" this.

bye
    michael

[Dmitry Timoshkov]

Frédéric Delanoy  wrote:
> > > I'm intending to convert dlls/kernel32/nls/fr.nls from CP 1252 to CP
> > > 65001 (UTF-8),
> >
> > Is there a ny reason behind that?
>
> Uniformisation? Universality? All other rc files (at least Fr ones,
> and most of the others) have already been
> converted.

If that's the only reason then there is no reasion to bother. On the other
hand if you are working on translating those files, or adding new translations
then you are free to choose the file encoding. dlls/kernel32/nls/* files
are autogenerated, so leave them alone.

[Frédéric Delanoy]

2010/5/31 Dmitry Timoshkov :
> Frédéric Delanoy  wrote:
>
> If that's the only reason then there is no reasion to bother. On the other
> hand if you are working on translating those files, or adding new translations
> then you are free to choose the file encoding. dlls/kernel32/nls/* files
> are autogenerated, so leave them alone.
>
> --
> Dmitry.

I've also fixed a translation problem.

Frédéric

[Frédéric Delanoy]

2010/5/31 Dmitry Timoshkov
> Frédéric Delanoy  wrote:
> > > > I'm intending to convert dlls/kernel32/nls/fr.nls from CP 1252 to CP
> > > > 65001 (UTF-8),
> > >
> > > Is there a ny reason behind that?
> >
> > Uniformisation? Universality? All other rc files (at least Fr ones,
> > and most of the others) have already been
> > converted.
>
> If that's the only reason then there is no reasion to bother. On the other
> hand if you are working on translating those files, or adding new
> translations
> then you are free to choose the file encoding. dlls/kernel32/nls/* files
> are autogenerated, so leave them alone.
>
> --
> Dmitry.

Autogenerated? From which source/using which mechanism?

If this is the case, shouldn't there be a comment in the file header that
says so?

Frédéric

[Dmitry Timoshkov]

Frédéric Delanoy  wrote:
> Autogenerated? From which source/using which mechanism?

Have a look at the file history.
> If this is the case, shouldn't there be a comment in the file header that
> says so?

Not necessarily. These data represent what Windows has, there is nothing
to change there.

[Alexandre Julliard]

Frédéric Delanoy  writes:
> Autogenerated? From which source/using which mechanism?

Actually it's not generated. Still, I'd suggest to leave it alone unless
there are things that need to be changed in there.

[Frédéric Delanoy]

2010/6/1 Alexandre Julliard
> Frédéric Delanoy  writes:
> > Autogenerated? From which source/using which mechanism?
>
> Actually it's not generated. Still, I'd suggest to leave it alone unless
> there are things that need to be changed in there.
>
> --
> Alexandre Julliard

In fact I was also fixing a spelling error (Arménian -> Arménien) and just
wondered
whether I could also convert it into UTF8 (M.Stefaniuc seemed to be OK with
that).

Or maybe it's safer to just fix the typo without converting character
encoding at the same time?

Frédéric

[Alexandre Julliard]

Frédéric Delanoy  writes:
> In fact I was also fixing a spelling error (Arménian -> Arménien) and just
> wondered
> whether I could also convert it into UTF8 (M.Stefaniuc seemed to be OK with
> that).
>
> Or maybe it's safer to just fix the typo without converting character
> encoding at the same time?

Yes, that's preferable. These files are more than translations, they are
data that many apps rely upon, so we need to be more conservative than
with plain rc files. Converting to utf-8 should wait until after 1.2.