Re: Piotr Caban : kernel32: Restore C locale for LC_CTYPE category.

[Jactry Zeng]

On Mon, Mar 18, 2019 at 9:24 PM Alexandre Julliard 
wrote:
> Module: wine
> Branch: master
> Commit: c8e195e1797bd513c91927743b3ed38fdfa86c2f
> URL:

https://source.winehq.org/git/wine.git/?a=commit;h=c8e195e1797bd513c91927743...
> Author: Piotr Caban 
> Date:   Mon Mar 18 20:03:40 2019 +0100
>
> kernel32: Restore C locale for LC_CTYPE category.

I found that this commit broke CJK input method on Wine, even with Sagawa's
patch for bug 46851[1].
It seems that Fcitx needs a LC_CTYPE matched to local language. Reverting
this commit and applying Sagawa's patch can fix the problem.
I don't know if we can fix the input medthod problem in imm32 or some other
components instead of reverting this commit.

[1] https://source.winehq.org/patches/data/161088

Piotr Caban : kernel32: Restore C locale for LC_CTYPE category.

[Piotr Caban]

Hi Jactry,

On 3/19/19 6:06 AM, Jactry Zeng wrote:
> On Mon, Mar 18, 2019 at 9:24 PM Alexandre Julliard <[e-mail] 
> > wrote:
> > Module: wine
> > Branch: master
> > Commit: c8e195e1797bd513c91927743b3ed38fdfa86c2f
> > URL:
>
> https://source.winehq.org/git/wine.git/?a=commit;h=c8e195e1797bd513c91927743...
> > Author: Piotr Caban <[e-mail]
>
> >
> > Date:   Mon Mar 18 20:03:40 2019 +0100
> >
> > kernel32: Restore C locale for LC_CTYPE category.
>
> I found that this commit broke CJK input method on Wine, even with 
> Sagawa's patch for bug 46851[1].
> It seems that Fcitx needs a LC_CTYPE matched to local language. 
> Reverting this commit and applying Sagawa's patch can fix the problem.
> I don't know if we can fix the input medthod problem in imm32 or some 
> other components instead of reverting this commit.

I'll look into it (I can reproduce it).

Thanks,
Piotr

Piotr Caban : kernel32: Restore C locale for LC_CTYPE category.

[Piotr Caban]

Hi,

I've looked for the solution of this problem and I failed to find one 
that works with C locale. It looks like XIM is not working when it's 
initialized in C or POSIX locale even if wchar_t or utf8 variant of 
lookup functions is used (XwcLookupString, Xutf8LookupString).

There are XIM bindings for XCB. Hopefully we will be able to use it in 
future. I'm not sure what is its status but it doesn't seem  to be 
widely available.

I see 2 possible solutions for that:
  - go with the ascii_{tolower,toupper,strcasecmp,strncasecmp} helpers 
and use them instead of glibc functions
  - try to detect locales that we don't support properly and set a 
different one in this case, I'm thinking about something like this:
if (strcasecmp("abc...z", "ABC...Z"))
     setlocale(LC_CTYPE, "en_US.UTF-8");
It will not work if en_US.UTF-8 locale is not available.

I prefer the first solution (it will work even if e.g. there's a library 
that changes locale). A simple grep shows that there are around 300 uses 
of tolower/toupper/strcasecmp/strncasecmp functions in wine (probably 
all will need to be changed).

Thanks,
Piotr

On 3/19/19 4:41 PM, Piotr Caban wrote:
> Hi Jactry,
>
> On 3/19/19 6:06 AM, Jactry Zeng wrote:
> > On Mon, Mar 18, 2019 at 9:24 PM Alexandre Julliard 
> > <[e-mail] > wrote:
> >  >
> >  > Module: wine
> >  > Branch: master
> >  > Commit: c8e195e1797bd513c91927743b3ed38fdfa86c2f
> >  > URL: 
> > https://source.winehq.org/git/wine.git/?a=commit;h=c8e195e1797bd513c91927743...
> >
> > >
> >  > Author: Piotr Caban <[e-mail] 
> > >
> >  > Date:   Mon Mar 18 20:03:40 2019 +0100
> >  >
> >  > kernel32: Restore C locale for LC_CTYPE category.
> >
> > I found that this commit broke CJK input method on Wine, even with 
> > Sagawa's patch for bug 46851[1].
> > It seems that Fcitx needs a LC_CTYPE matched to local language. 
> > Reverting this commit and applying Sagawa's patch can fix the problem.
> > I don't know if we can fix the input medthod problem in imm32 or some 
> > other components instead of reverting this commit.
>
> I'll look into it (I can reproduce it).
>
> Thanks,
> Piotr

Piotr Caban : kernel32: Restore C locale for LC_CTYPE category.

[Ken Thomases]

On Mar 20, 2019, at 8:28 AM, Piotr Caban  wrote:
> I've looked for the solution of this problem and I failed to find one that works with C locale. It looks like XIM is not working when it's initialized in C or POSIX locale even if wchar_t or utf8 variant of lookup functions is used (XwcLookupString, Xutf8LookupString).

[…]
> I see 2 possible solutions for that:
> - go with the ascii_{tolower,toupper,strcasecmp,strncasecmp} helpers and use them instead of glibc functions

What about the xlocale functions (e.g. tolower_l())?  If you pass a NULL locale_t to them, they operate in the C locale.

-Ken

Piotr Caban : kernel32: Restore C locale for LC_CTYPE category.

[Chip Davis]

March 20, 2019 9:58 AM, "Ken Thomases"  wrote:
> On Mar 20, 2019, at 8:28 AM, Piotr Caban  wrote:
> > I've looked for the solution of this problem and I failed to find one that works with C locale. It
> > looks like XIM is not working when it's initialized in C or POSIX locale even if wchar_t or utf8
> > variant of lookup functions is used (XwcLookupString, Xutf8LookupString).
>
> […]
> > I see 2 possible solutions for that:
> > - go with the ascii_{tolower,toupper,strcasecmp,strncasecmp} helpers and use them instead of glibc
> >
> > functions
>
> What about the xlocale functions (e.g. tolower_l())? If you pass a NULL locale_t to them, they
> operate in the C locale.

Only the POSIX ones do that. The Microsoft ones use the global locale. Also, I suspect those functions aren't available everywhere--even though POSIX 2008 has been a thing for over a decade.
> -Ken

Chip

Piotr Caban : kernel32: Restore C locale for LC_CTYPE category.

[Ken Thomases]

On Mar 20, 2019, at 10:02 AM, Chip Davis  wrote:
> March 20, 2019 9:58 AM, "Ken Thomases"  wrote:
> > On Mar 20, 2019, at 8:28 AM, Piotr Caban  wrote:
> > > I've looked for the solution of this problem and I failed to find one that works with C locale. It
> > > looks like XIM is not working when it's initialized in C or POSIX locale even if wchar_t or utf8
> > > variant of lookup functions is used (XwcLookupString, Xutf8LookupString).
> >
> > […]
> > > I see 2 possible solutions for that:
> > > - go with the ascii_{tolower,toupper,strcasecmp,strncasecmp} helpers and use them instead of glibc
> > >
> > > functions
> >
> > What about the xlocale functions (e.g. tolower_l())? If you pass a NULL locale_t to them, they
> > operate in the C locale.
>
> Only the POSIX ones do that. The Microsoft ones use the global locale. Also, I suspect those functions aren't available everywhere--even though POSIX 2008 has been a thing for over a decade.

But aren't we only interested in changing the behavior of the POSIX ones?

An alternative approach might be to leave LC_CTYPE set to the C locale but wrap the XIM functions in uselocale() calls to set the locale to the one that Wine had been using before this change and then set it back.  I don't know if that will be enough to get things to work.

-Ken

Piotr Caban : kernel32: Restore C locale for LC_CTYPE category.

[Piotr Caban]

On 3/20/19 4:25 PM, Ken Thomases wrote:
> On Mar 20, 2019, at 10:02 AM, Chip Davis  wrote:
> > March 20, 2019 9:58 AM, "Ken Thomases"  wrote:
> > > On Mar 20, 2019, at 8:28 AM, Piotr Caban  wrote:
> > > > I've looked for the solution of this problem and I failed to find one that works with C locale. It
> > > > looks like XIM is not working when it's initialized in C or POSIX locale even if wchar_t or utf8
> > > > variant of lookup functions is used (XwcLookupString, Xutf8LookupString).
> > >
> > > […]
> > > > I see 2 possible solutions for that:
> > > > - go with the ascii_{tolower,toupper,strcasecmp,strncasecmp} helpers and use them instead of glibc
> > > >
> > > > functions
> > >
> > > What about the xlocale functions (e.g. tolower_l())? If you pass a NULL locale_t to them, they
> > > operate in the C locale.
> >
> > Only the POSIX ones do that. The Microsoft ones use the global locale. Also, I suspect those functions aren't available everywhere--even though POSIX 2008 has been a thing for over a decade.
>
> But aren't we only interested in changing the behavior of the POSIX ones?

Then we will need to provide tolower_l implementation for systems that 
don't support it (unless you want to use it inside of the ascii_tolower 
function, we will also need to include config.h in many files).
> An alternative approach might be to leave LC_CTYPE set to the C locale but wrap the XIM functions in uselocale() calls to set the locale to the one that Wine had been using before this change and then set it back.  I don't know if that will be enough to get things to work.

XIM is ignoring locale set with uselocale function. You will need to 
call setlocale to make it work but it changes the locale for whole process.

For the reference, the functions that needs to be called in non C locale 
are:
XOpenIM
XSupportsLocale
XSetLocaleModifiers

Thanks,
Piotr

Piotr Caban : kernel32: Restore C locale for LC_CTYPE category.

[Ken Thomases]

On Mar 20, 2019, at 11:16 AM, Piotr Caban  wrote:
> On 3/20/19 4:25 PM, Ken Thomases wrote:
> > An alternative approach might be to leave LC_CTYPE set to the C locale but wrap the XIM functions in uselocale() calls to set the locale to the one that Wine had been using before this change and then set it back.  I don't know if that will be enough to get things to work.
>
> XIM is ignoring locale set with uselocale function. You will need to call setlocale to make it work but it changes the locale for whole process.

Then can we leave the global locale as dictated by environment variables and do uselocale() for all Wine-created threads to make them use the C locale?

-Ken

Piotr Caban : kernel32: Restore C locale for LC_CTYPE category.

[Piotr Caban]

On 3/20/19 5:55 PM, Ken Thomases wrote:
> Then can we leave the global locale as dictated by environment variables and do uselocale() for all Wine-created threads to make them use the C locale?

That would work (at least for XIM). I'm not sure if it's a nicer 
solution. I'll send a patch with that change to show how this code may 
look like.

Thanks,
Piotr