Re: msvcrt: Implement _mbctombb.

[Maarten Lankhorst]

Hi David,

David Hedberg wrote:
> On Wed, Feb 24, 2010 at 7:12 AM, Maarten Lankhorst
>  wrote:
> > Hi David,
>
> Hi Maarten,
> > David Hedberg wrote:
> > > +/* Maps multibyte cp932 punctuation marks to single byte equivalents */
> > > +static unsigned int mbctombb_932_punct[] = {
> > > -
> > >
> > > 0x20,0xa4,0xa1,0x2c,0x2e,0xa5,0x3a,0x3b,0x3f,0x21,0xde,0xdf,0x00,0x00,0x00,0x5e,
> > > -
> > >
> > > 0x7e,0x5f,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xb0,0x00,0x00,0x2f,0x00,
> > > -
> > >
> > > 0x00,0x00,0x7c,0x00,0x00,0x60,0x27,0x00,0x22,0x28,0x29,0x00,0x00,0x5b,0x5d,0x7b,
> > > -
> > >
> > > 0x7d,0x00,0x00,0x00,0x00,0xa2,0xa3,0x00,0x00,0x00,0x00,0x2b,0x2d,0x00,0x00,0x00,
> > > -
> > >
> > > 0x00,0x3d,0x00,0x3c,0x3e,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x5c,
> > > - 0x24,0x00,0x00,0x25,0x23,0x26,0x2a,0x40};
> > > -
> > >
> > > +/* Maps multibyte cp932 hiragana/katakana to single-byte equivalents */
> > > +static unsigned int mbctombb_932_kana[] = {
> > > -
> > >
> > > 0xa7,0xb1,0xa8,0xb2,0xa9,0xb3,0xaa,0xb4,0xab,0xb5,0xb6,0xb6,0xb7,0xb7,0xb8,0xb8,
> > > -
> > >
> > > 0xb9,0xb9,0xba,0xba,0xbb,0xbb,0xbc,0xbc,0xbd,0xbd,0xbe,0xbe,0xbf,0xbf,0xc0,0xc0,
> > > -
> > >
> > > 0xc1,0xc1,0xaf,0xc2,0xc2,0xc3,0xc3,0xc4,0xc4,0xc5,0xc6,0xc7,0xc8,0xc9,0xca,0xca,
> > > -
> > >
> > > 0xca,0xcb,0xcb,0xcb,0xcc,0xcc,0xcc,0xcd,0xcd,0xcd,0xce,0xce,0xce,0xcf,0xd0,0xd1,
> > > -
> > >
> > > 0xd2,0xd3,0xac,0xd4,0xad,0xd5,0xae,0xd6,0xd7,0xd8,0xd9,0xda,0xdb,0xdc,0xdc,0xb2,
> > > - 0xb4,0xa6,0xdd,0xb3,0xb6,0xb9};
> >
> > You're hardcoding this for a single code page? Are you sure you cannot use a
> > more generic function instead of hardcoding a table?
>
> I haven't found anything that might handle it. The function in
> question maps the characters in a somewhat "irregular fashion", and
> only makes sense for code page 932. There's a static function
> "han2zen" in dlls/mlang/mlang.c that appears to be the "inverse" (to
> the extent this function is reversible), that one too is implemented
> with a couple of tables.

Well, it seems msdn says the function used to be called zentohan, so if 
the function only makes sense for cp 932 then adding the table is 
harmless, I was afraid that more locales would need that function, in 
which case a more generic solution would have been preferred.

Cheers,
Maarten.

PS: Use 'reply all' so wine-devel gets cc'd.

msvcrt: Implement _mbctombb.

[David Hedberg]

Hi again,

On Wed, Feb 24, 2010 at 8:27 AM, Maarten Lankhorst
 wrote:
> Hi David,
>
> David Hedberg wrote:
> > On Wed, Feb 24, 2010 at 7:12 AM, Maarten Lankhorst
> >  wrote:
> > > Hi David,
> >
> > Hi Maarten,
> > > David Hedberg wrote:
> > > > +/* Maps multibyte cp932 punctuation marks to single byte equivalents */
> > > > +static unsigned int mbctombb_932_punct[] = {
> > > > -
> > > >
> > > > 0x20,0xa4,0xa1,0x2c,0x2e,0xa5,0x3a,0x3b,0x3f,0x21,0xde,0xdf,0x00,0x00,0x00,0x5e,
> > > > -
> > > >
> > > > 0x7e,0x5f,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xb0,0x00,0x00,0x2f,0x00,
> > > > -
> > > >
> > > > 0x00,0x00,0x7c,0x00,0x00,0x60,0x27,0x00,0x22,0x28,0x29,0x00,0x00,0x5b,0x5d,0x7b,
> > > > -
> > > >
> > > > 0x7d,0x00,0x00,0x00,0x00,0xa2,0xa3,0x00,0x00,0x00,0x00,0x2b,0x2d,0x00,0x00,0x00,
> > > > -
> > > >
> > > > 0x00,0x3d,0x00,0x3c,0x3e,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x5c,
> > > > - 0x24,0x00,0x00,0x25,0x23,0x26,0x2a,0x40};
> > > > -
> > > >
> > > > +/* Maps multibyte cp932 hiragana/katakana to single-byte equivalents */
> > > > +static unsigned int mbctombb_932_kana[] = {
> > > > -
> > > >
> > > > 0xa7,0xb1,0xa8,0xb2,0xa9,0xb3,0xaa,0xb4,0xab,0xb5,0xb6,0xb6,0xb7,0xb7,0xb8,0xb8,
> > > > -
> > > >
> > > > 0xb9,0xb9,0xba,0xba,0xbb,0xbb,0xbc,0xbc,0xbd,0xbd,0xbe,0xbe,0xbf,0xbf,0xc0,0xc0,
> > > > -
> > > >
> > > > 0xc1,0xc1,0xaf,0xc2,0xc2,0xc3,0xc3,0xc4,0xc4,0xc5,0xc6,0xc7,0xc8,0xc9,0xca,0xca,
> > > > -
> > > >
> > > > 0xca,0xcb,0xcb,0xcb,0xcc,0xcc,0xcc,0xcd,0xcd,0xcd,0xce,0xce,0xce,0xcf,0xd0,0xd1,
> > > > -
> > > >
> > > > 0xd2,0xd3,0xac,0xd4,0xad,0xd5,0xae,0xd6,0xd7,0xd8,0xd9,0xda,0xdb,0xdc,0xdc,0xb2,
> > > > - 0xb4,0xa6,0xdd,0xb3,0xb6,0xb9};
> > >
> > > You're hardcoding this for a single code page? Are you sure you cannot
> > > use a
> > > more generic function instead of hardcoding a table?
> >
> > I haven't found anything that might handle it. The function in
> > question maps the characters in a somewhat "irregular fashion", and
> > only makes sense for code page 932. There's a static function
> > "han2zen" in dlls/mlang/mlang.c that appears to be the "inverse" (to
> > the extent this function is reversible), that one too is implemented
> > with a couple of tables.
>
> Well, it seems msdn says the function used to be called zentohan, so if the
> function only makes sense for cp 932 then adding the table is harmless, I
> was afraid that more locales would need that function, in which case a more
> generic solution would have been preferred.

Well, I should add that it only makes sense for cp 932 as far as I
know, but then I'm only familiar with Japanese. More important perhaps
is that I also ran some tests on Win XP and Win 7 trying to get it to
return anything for other code pages (basically looped through them
all with _setmbcp), but only with cp 932 did it return anything other
than what was given to it.

I'm no expert on the win32 API though, so it is possible that I have
missed something. :-)
> Cheers,
> Maarten.
>
> PS: Use 'reply all' so wine-devel gets cc'd.

Woops, sorry!
>

Thanks,
David

msvcrt: Implement _mbctombb.

[Reece Dunn]

On 24 February 2010 08:51, David Hedberg  wrote:
> Well, I should add that it only makes sense for cp 932 as far as I
> know, but then I'm only familiar with Japanese. More important perhaps
> is that I also ran some tests on Win XP and Win 7 trying to get it to
> return anything for other code pages (basically looped through them
> all with _setmbcp), but only with cp 932 did it return anything other
> than what was given to it.
>
> I'm no expert on the win32 API though, so it is possible that I have
> missed something. :-)

Could you add that to your test case, and then run it on the
winetestbot with a range of OSs?

This will help show that it is only CP 932 that is affected by that function.

Thanks,
- Reece