[Bug 33447] New: Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

Bug #: 33447
           Summary: Wine GUI on Mac display Traditional Chinese instead
                    when host language is Simplify Chinese
           Product: Wine
           Version: 1.5.28
          Platform: x86
        OS/Version: Mac OS X
            Status: NEW
          Severity: normal
          Priority: P2
         Component: -unknown
        AssignedTo: [e-mail]
        ReportedBy: [e-mail]
    Classification: Unclassified

Step to reproduce:

1. Install Wine from macport, or build from wine-git

2. Create a fresh MacOS user, set Mac UI language to Simplified Chinese.
$ locale
LANG="zh_CN.UTF-8"
LC_COLLATE="zh_CN.UTF-8"
LC_CTYPE="zh_CN.UTF-8"
LC_MESSAGES="zh_CN.UTF-8"
LC_MONETARY="zh_CN.UTF-8"
LC_NUMERIC="zh_CN.UTF-8"
LC_TIME="zh_CN.UTF-8"
LC_ALL=

3. start winecfg/regedit/notepad

Expect result:
The default GUI language should be Simplified Chinese

Actual result:
The default GUI language is Traditional Chinese. I've tried many way to
modified locale related environment value, but still can't find any way to
display Simplified Chinese in Wine GUI. locale things on Mac seems different as
Linux.
+font log show that Wine do have the correct codepage (936 for Simplified
Chinese, 950 for Traditional Chinese), not sure why it still display
Traditional Chinese.

Will attach some screenshot.

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

Qian Hong  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |download, source

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #1 from Qian Hong  2013-04-23 10:21:19 CDT ---
Created attachment 44266
  --> http://bugs.winehq.org/attachment.cgi?id=44266
Screenshot: Wine notepad menu display Traditional Chinese

The menu is displayed in Traditional Chinese, the content in notepad are the
correct Simplified Chinese version of those menu entries.

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #2 from Bruno Jesus  2013-04-23 11:04:25 CDT ---
Qian, can't you attach the image as png/jpg? Also, it would be interesting to
make a comparison between what is the expected and what is the bad image
(side-by-side) so developers can see the correct version.

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #3 from Qian Hong  2013-04-23 11:10:24 CDT ---
(In reply to comment #2)
> Qian, can't you attach the image as png/jpg? Also, it would be interesting to
> make a comparison between what is the expected and what is the bad image
> (side-by-side) so developers can see the correct version.

Hmm... My Mac screenshot application generate tiff by default, I'll convert it
to png on my Linux...

I can't workaround the bug on Mac at all, that's why I didn't upload a 'good
image'. I'll upload a good image from Linux instead.

Thanks for reminder :)

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #4 from Qian Hong  2013-04-23 11:15:26 CDT ---
Created attachment 44269
  --> http://bugs.winehq.org/attachment.cgi?id=44269
Screenshot: bad result on Mac (traditional chinese)

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #5 from Qian Hong  2013-04-23 11:16:09 CDT ---
Created attachment 44270
  --> http://bugs.winehq.org/attachment.cgi?id=44270
Screenshot: good result on Linux (simplified chinese)

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

Qian Hong  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #44266|0                           |1
        is obsolete|                            |

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

Qian Hong  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #44269|0                           |1
        is obsolete|                            |

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #6 from Qian Hong  2013-04-23 11:23:41 CDT ---
Unicode of those Chinese:

Bad: (Traditional Chinese)
檔案:  U+6A94 U+6848
編輯:  U+7DE8 U+8F2F
搜尋:  U+641C U+5C0B
求助:  U+6C42 U+52A9

Good: (Simplified Chinese)
文件:  U+6587 U+4EF6
编辑:  U+7F16 U+8F91
搜索:  U+641C U+7D22
帮助:  U+5E2E U+52A9

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

Qian Hong  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #44269|1                           |0
        is obsolete|                            |

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

Ken Thomases  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #7 from Ken Thomases  2013-04-23 20:54:54 CDT ---
What is selected for the region in System Preferences > Language & Text >
Region (Formats on 10.6.x)?

Also, please attach a +nls log.

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #8 from Qian Hong  2013-04-24 02:18:48 CDT ---
Created attachment 44278
  --> http://bugs.winehq.org/attachment.cgi?id=44278
Log: start notepad with +nls trace

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #9 from Qian Hong  2013-04-24 02:26:52 CDT ---
(In reply to comment #7)
> What is selected for the region in System Preferences > Language & Text >
> Region (Formats on 10.6.x)?

The language order is:
Simplified Chinese (简体中文), English, Japanese(日本语), etc...

The Region is set to China  （中国）

Simplified Chinese is used in China (中国), Traditional Chinese is used in Taiwan
(台湾)

Will attach some screenshot.

Thanks.

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #10 from Qian Hong  2013-04-24 02:30:01 CDT ---
Created attachment 44280
  --> http://bugs.winehq.org/attachment.cgi?id=44280
Screenshot: language settings

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #11 from Qian Hong  2013-04-24 02:30:54 CDT ---
Created attachment 44281
  --> http://bugs.winehq.org/attachment.cgi?id=44281
Screenshot: region settings

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #12 from Qian Hong  2013-04-24 02:32:17 CDT ---
BTW, I'm testing on OS X 10.8.3

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #13 from Qian Hong  2013-04-24 13:54:59 CDT ---
Hello,

In dlls/kernel32/locale.c:LOCALE_Init() line 3240:

between #ifdef __APPLE__ and #endif:
parse_locale_name( buffer, &locale_name );

On Simplified Chinese MacOS, buffer is "zh_Hans", here 'zh' means ‘中’，Han means
‘汉’， ‘s’ means ‘Simplified’. For Traditional Chinese MacOS, the similar string
should be "zh_Hant". Currently parse_locale_name() doesn't have knowledge about
"zh_Hans" or "zh_Hant", it only know "zh-CN"/"zh-TW"/etc.

As result, parse_locale_name incorrectly set locale_name.lcid to 0x404, that's
Traditional Chinese's LCID. That is why GetSystemDefaultLangID() returns 0x804
(Simplified Chinese) while GetUserDefaultLangID() returns 0x404. Currently we
choose default GUI language depends on GetUserDefaultLangID(), so Wine GUI on
Mac display Traditional Chinese even on Simplified Chinese host system.

I'll try to write a fix for it.

Cheers.

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #14 from Ken Thomases  2013-04-24 14:33:49 CDT ---
The Mac has language settings and region (locale) settings.  In
dlls/kernel32/locale.c:LOCALE_Init(), Wine uses the CFLocale API to access the
latter. It gets separate language and country codes and then combines them into
the form of a POSIX locale identifier.  In your log that looks like:

trace:nls:LOCALE_Init setting locale to 'zh_CN.UTF-8'

So, that's correct.

Later, it tries to figure out the system language setting.  It only does this
if neither LC_ALL or LC_MESSAGES are set in the environment.  They aren't in
your case.  Your language setting is "zh-Hans", which is an IETF / BCP 47
language tag.  The Mac API seems to have attempted to canonicalize that from
into a locale identifier and got "zh_Hans".  (My Mac OS X 10.6.x system doesn't
do that and leaves it as "zh-Hans", although the results aren't any better.)

trace:nls:LOCALE_Init setting lcid_LC_MESSAGES to 'zh_Hans'

We need a way to convert the language tag (combines a language subtag and a
script subtag) to a locale identifier (combines a language subtag and a country
code).  There's nothing in the Mac APIs that does this, as far as I'm aware.  I
found some code in gettext that deals with this problem with hard-coded lookup
tables (although technically, it's still working with locale identifiers).
http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-runtime/intl/local...

As an aside, the LOCALE_Init() code for determining the preferred language from
the Mac system settings could be simplified.  We can use the
CFLocaleCopyPreferredLanguages() function rather than using CFBundle stuff.

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #15 from Ken Thomases  2013-04-24 14:49:21 CDT ---
Rather than attempting to change parse_locale_name(), maybe a better approach
would be to implement a separate parsing path specific to Mac language tags. 
It would expect a language subtag ("zh" in this case) and optionally a hyphen
("-") followed by a script code ("Hans").  Then, it could enumerate the locales
and compare the language subtag with LOCALE_SISO639LANGNAME and the script code
with LOCALE_SSCRIPTS.  LOCALE_SSCRIPTS is a list of script codes with semicolon
separators and terminator, so we'd want to check if the Mac script code was an
element of the list.

A potential problem is that multiple Windows locales might match.  I don't know
if the Windows locales are in a priority order so that we can rely on the first
match being the best match.

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #16 from Qian Hong  2013-04-29 12:44:55 CDT ---
Created attachment 44330
  --> http://bugs.winehq.org/attachment.cgi?id=44330
Patch: partial fix

Hi Ken,

Thanks a lot for your great analysis.
I found the multiple-Windows-locales-match issue is tricky, it seems there is
no really better way rather than some hardcode work.

Rather than fix it completely right now, I'd like to send a partial fix first.
I'm attaching a patch which fix the case when user language setting is matching
with user region setting, but haven't fix the case when user language setting
is not matching with user region setting. This patch use the CoreFoundation API
CFLocaleCreateCanonicalLanguageIdentifierFromString which will automatically
convert 'zh_CN/zh_SG' to 'zh-Hans', convert 'zh_HK/zh_TW/zh_MO' to 'zh-Hant'.

I appreciate for any comments on it, if you don't object to it I'll send it to
wine-patch first, and leave this bug open until we have decide if hardcoding is
the right way for completely fix this bug.

Thanks a lot!

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

Qian Hong  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #44330|0                           |1
           is patch|                            |

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

Qian Hong  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |patch
          Component|-unknown                    |kernel32

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #17 from Ken Thomases  2013-04-29 13:31:30 CDT ---
Hi Qian,

That approach does seem like it will help in this case, although, as you note,
not in all cases.

You leak the objected created by
CFLocaleCreateCanonicalLanguageIdentifierFromString(), though.  In general,
Core Foundation functions with "Create" or "Copy" in their names return an
object that the caller is responsible for releasing.

Also, I wouldn't bother with the new user_locale_string_utf8_ref variable. 
Instead, just get the C string from user_locale_string_ref into user_locale and
then strcat ".UTF-8" onto it.

Thanks for your work on this!

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #18 from Qian Hong  2013-04-29 14:10:10 CDT ---
Created attachment 44332
  --> http://bugs.winehq.org/attachment.cgi?id=44332
Patch: partial fix v2

(In reply to comment #17)
> Hi Qian,
> You leak the objected created by
> CFLocaleCreateCanonicalLanguageIdentifierFromString(), though.  In general,
> Core Foundation functions with "Create" or "Copy" in their names return an
> object that the caller is responsible for releasing.

Aha, good catch, thanks!
> Also, I wouldn't bother with the new user_locale_string_utf8_ref variable. 
> Instead, just get the C string from user_locale_string_ref into user_locale and
> then strcat ".UTF-8" onto it.

Good idea, thanks.

Thanks a lot for review, how about this one?

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #19 from Ken Thomases  2013-04-29 14:22:39 CDT ---
Looks better.  One more small suggestion while I'm being picky: it's probably
best to rename preferred_locales to preferred_langs now, since that's what it
really is.

Oh, and also, we no longer need to #include <CoreFoundation/CFBundle.h> since
we're not using CFBundleCopyLocalizationsForPreferences().

Thanks again.

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #20 from Qian Hong  2013-04-29 14:48:40 CDT ---
Created attachment 44333
  --> http://bugs.winehq.org/attachment.cgi?id=44333
Patch: partial fix v3

Thanks Ken again, good catch!

Seems like this one is ok but any picky is welcome :)

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #21 from Ken Thomases  2013-04-29 15:25:00 CDT ---
Looks good to me.

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

http://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #22 from Bruno Jesus  2013-04-30 12:54:38 CDT ---
Patch commited:
http://source.winehq.org/git/wine.git/?a=commit;h=32e5314250db3cbf37d414849f...

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplify Chinese

https://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #23 from Nikolay Sivov  ---
Hi, Qian.

Your patch was committed, does it mean we can close this?

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplified Chinese

https://bugs.winehq.org/show_bug.cgi?id=33447

Nikolay Sivov  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Wine GUI on Mac display     |Wine GUI on Mac display
                   |Traditional Chinese instead |Traditional Chinese instead
                   |when host language is       |when host language is
                   |Simplify Chinese            |Simplified Chinese

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplified Chinese

https://bugs.winehq.org/show_bug.cgi?id=33447

Jactry Zeng  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplified Chinese

https://bugs.winehq.org/show_bug.cgi?id=33447

--- Comment #24 from Jactry Zeng  ---
Created attachment 50956
  --> https://bugs.winehq.org/attachment.cgi?id=50956
wine notepad on osx

Hi Nikolay,
(In reply to Nikolay Sivov from comment #23)
> Hi, Qian.
>
> Your patch was committed, does it mean we can close this?

Current wine works fine for me.

OS X 10.9.5:
LANG="zh_CN.UTF-8"
LC_COLLATE="zh_CN.UTF-8"
LC_CTYPE="zh_CN.UTF-8"
LC_MESSAGES="zh_CN.UTF-8"
LC_MONETARY="zh_CN.UTF-8"
LC_NUMERIC="zh_CN.UTF-8"
LC_TIME="zh_CN.UTF-8"
LC_ALL=

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplified Chinese

https://bugs.winehq.org/show_bug.cgi?id=33447

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #25 from Austin English  ---
(In reply to Jactry Zeng from comment #24)
> Created attachment 50956 [details]
> wine notepad on osx
>
> Hi Nikolay,
> (In reply to Nikolay Sivov from comment #23)
> > Hi, Qian.
> >
> > Your patch was committed, does it mean we can close this?
>
> Current wine works fine for me.
>
> OS X 10.9.5:
> LANG="zh_CN.UTF-8"
> LC_COLLATE="zh_CN.UTF-8"
> LC_CTYPE="zh_CN.UTF-8"
> LC_MESSAGES="zh_CN.UTF-8"
> LC_MONETARY="zh_CN.UTF-8"
> LC_NUMERIC="zh_CN.UTF-8"
> LC_TIME="zh_CN.UTF-8"
> LC_ALL=

Fixed.

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplified Chinese

https://bugs.winehq.org/show_bug.cgi?id=33447

Nikolay Sivov  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
      Fixed by SHA1|                            |32e5314250db3cbf37d414849f6
                   |                            |3a5a5df0ca9b6

[Bug 33447] Wine GUI on Mac display Traditional Chinese instead when host language is Simplified Chinese

https://bugs.winehq.org/show_bug.cgi?id=33447

Alexandre Julliard  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #26 from Alexandre Julliard  ---
Closing bugs fixed in 1.7.39.