Re: configure: add ncursesw detecting [try 2]

[Alexandre Julliard]

Vitaly Lipatov  writes:
> Try 2, with ncursesw/ncurses detecting
> In order to build wineconsole run properly in utf8 locale, we use ncursesw, 
> which can handle wchar strings.

Why would you need wchar strings if you use utf-8?  Have you
configured your locale properly?

configure: add ncursesw detecting [try 2]

[Vitaly Lipatov]

В сообщении от 15 декабря 2006 13:39 Alexandre Julliard написал(a):
> Vitaly Lipatov  writes:
> > Try 2, with ncursesw/ncurses detecting
> > In order to build wineconsole run properly in utf8 locale, we use
> > ncursesw, which can handle wchar strings.
>
> Why would you need wchar strings if you use utf-8?  Have you
> configured your locale properly?

We can't get correct non latin output in utfed console with normal ncurses. As 
far as I know there is some problems in normal ncurses with utf8 (multbyte 
symbols)... Only ncursesw supports utf8 output correctly (and wide char)... I 
am wrong?

Follow is example code with try to out cyrillic text. If we run it utf8 
locale, only printf prints correct letters.

#include <ncurses.h>
#include <stdio.h>
#include <string.h>

int main()
{
   setlocale(0,"");
    int i=0;
   unsigned char cOut[] = {0xd0, 0x9f, 0xd1, 0x80, 0xd0, 0xb8, 0xd0, 0xb2, 
0xd0,0xb5,0xd1, 0x82,0x00 };

printf("String with cyrillic symbols (out with printf):\n");
   printf("%s\n",cOut);

initscr();

mvaddstr(2,0,"String with cyrillic symbols:");

mvaddstr(4,0,cOut);
   refresh();
   getch();
   endwin();

return 0;
}

configure: add ncursesw detecting [try 2]

[Alexandre Julliard]

Vitaly Lipatov  writes:
> We can't get correct non latin output in utfed console with normal ncurses. As 
> far as I know there is some problems in normal ncurses with utf8 (multbyte 
> symbols)... Only ncursesw supports utf8 output correctly (and wide char)... I 
> am wrong?

I don't know, maybe not, curses isn't exactly a great API...

In any case, you'll need to send the rest of the code, I'd like to see
how you are using it before adding configure checks.

configure: add ncursesw detecting [try 2]

[Anatoly Lyutin]

Alexandre Julliard wrote:
> Vitaly Lipatov  writes:
> > We can't get correct non latin output in utfed console with normal ncurses. As 
> > far as I know there is some problems in normal ncurses with utf8 (multbyte 
> > symbols)... Only ncursesw supports utf8 output correctly (and wide char)... I 
> > am wrong?
>
> I don't know, maybe not, curses isn't exactly a great API...
>
> In any case, you'll need to send the rest of the code, I'd like to see
> how you are using it before adding configure checks.

This patch fixs wineconsole for non latin symbols. If we have ncursesw 
we use unicode functions to console output.

configure: add ncursesw detecting [try 2]

[Dmitry Timoshkov]

"Anatoly Lyutin"  wrote:
> This patch fixs wineconsole for non latin symbols. If we have ncursesw 
> we use unicode functions to console output.

Most likely the real fixes in your patch are changing CP_ACP to CP_UNIXCP
and getting rid of hardcoded vkkeyscan_table. Do you have an evidence that
ncurses is so much broken that it really can't handle UTF-8? Does it work
if you change the locale to a not UTF-8 one?

configure: add ncursesw detecting [try 2]

[Vitaly Lipatov]

В сообщении от 5 января 2007 05:33 Dmitry Timoshkov написал(a):
> "Anatoly Lyutin"  wrote:
> > This patch fixs wineconsole for non latin symbols. If we have ncursesw
> > we use unicode functions to console output.
>
> Most likely the real fixes in your patch are changing CP_ACP to CP_UNIXCP
> and getting rid of hardcoded vkkeyscan_table. Do you have an evidence that
> ncurses is so much broken that it really can't handle UTF-8? Does it work
> if you change the locale to a not UTF-8 one?

Dmitry, as I already post in this thread, we have test program which does not 
print correctly ever in UTF-8 locale.
Usual ncurses works in 8-bit locales like koi8-r correctly, but we can't 
output in utf8 locale.

Test program for output utf8 string in utf-8 console:

#include <ncurses.h>
#include <stdio.h>
#include <string.h>
#include <locale.h>

int main()
{
   setlocale(LC_ALL,"");
    int i=0;
   unsigned char cOut[] = {0xd0, 0x9f, 0xd1, 0x80, 0xd0, 0xb8, 0xd0, 0xb2, 
0xd0,0xb5,0xd1, 0x82,0x00 };

printf("String with cyrillic symbols (out with printf):\n");
   printf("%s\n",cOut);    
   
    initscr();
   
   mvaddstr(2,0,"String with cyrillic symbols:");

mvaddstr(4,0,cOut);
   refresh();
   getch();
   endwin();

return 0;
}

configure: add ncursesw detecting [try 2]

[Dmitry Timoshkov]

"Vitaly Lipatov"  wrote:
> Dmitry, as I already post in this thread, we have test program which does not 
> print correctly ever in UTF-8 locale.
> Usual ncurses works in 8-bit locales like koi8-r correctly, but we can't 
> output in utf8 locale.

According to http://www.gnu.org/software/ncurses/ncurses.html ncurses does
support UTF-8, probably you need to update to a newer version.

configure: add ncursesw detecting [try 2]

[Dmitry Timoshkov]

"Dmitry Timoshkov"  wrote:
> "Vitaly Lipatov"  wrote:
> > Dmitry, as I already post in this thread, we have test program which does not 
> > print correctly ever in UTF-8 locale.
> > Usual ncurses works in 8-bit locales like koi8-r correctly, but we can't 
> > output in utf8 locale.
>
> According to http://www.gnu.org/software/ncurses/ncurses.html ncurses does
> support UTF-8, probably you need to update to a newer version.

Actually ncurses-5.6/README states:

"If you configure using the --enable-widec option, a "w" is appended to the
library names (e.g., libncursesw.a), and the resulting libraries support
wide-characters, e.g., via a UTF-8 locale.  The corresponding header files
are compatible with the non-wide-character configuration; wide-character
features are provided by ifdef's in the header files.  The wide-character
library interfaces are not binary-compatible with the non-wide-character
version."

So, you are right Vitaly, "normal" ncurses library doesn't support UTF-8,
they have invented libncursesw instead of adding a proper UTF-8 support.
But looks like all they redefine all existing APIs to 'w' versions. so
you don't need to call 'w' versions such as wget_wch directly, get_wch
will be redefined to wget_wch instead.

configure: add ncursesw detecting [try 2]

[Vitaly Lipatov]

On Tuesday 09 January 2007 10:18, Dmitry Timoshkov wrote:
...
> So, you are right Vitaly, "normal" ncurses library doesn't
> support UTF-8, they have invented libncursesw instead of
> adding a proper UTF-8 support. But looks like all they
> redefine all existing APIs to 'w' versions. so you don't need
> to call 'w' versions such as wget_wch directly, get_wch will
> be redefined to wget_wch instead.

We try to rewrote code in universal manner, but old curses code 
in wineconsole had many hacks and we are changed some parts 
anywhere.
I need to accept this patch for detecting ncursesw. Then we can 
send next patches for unicoding.
I do not know about function redefining, there are incompatible 
function's arguments (cchar_t vs wchar_t) and it needs various 
code...
As I see we need use libncursesw as the main curses library, and 
have old curses support for compatibility with some old systems.

configure: add ncursesw detecting [try 2]

[Dmitry Timoshkov]

"Vitaly Lipatov"  wrote:
> We try to rewrote code in universal manner, but old curses code 
> in wineconsole had many hacks and we are changed some parts 
> anywhere.
> I need to accept this patch for detecting ncursesw. Then we can 
> send next patches for unicoding.

Your patch introduces a lot of #ifdef's, that's not something
to be easily accepted. According to the info I quoted an old code
is source compatible with ncursesw, so you shouldn't need #ifdefs
at all, at least in theory.
> I do not know about function redefining, there are incompatible 
> function's arguments (cchar_t vs wchar_t) and it needs various 
> code...

Then existing code needs to be fixed to be compatible with old and
new (redefined) interfaces.
> As I see we need use libncursesw as the main curses library, and 
> have old curses support for compatibility with some old systems.

That's why fixing existing code is preferred to changing it and
potentially breaking compatibility with normal ncurses.

configure: add ncursesw detecting [try 2]

[Dmitry Timoshkov]

"Vitaly Lipatov"  wrote:
> As I see we need use libncursesw as the main curses library, and 
> have old curses support for compatibility with some old systems.

Forgot to ask, what application are you trying to fix? Is it freely
downloadable?

configure: add ncursesw detecting [try 2]

[Vitaly Lipatov]

On Tuesday 09 January 2007 12:14, Dmitry Timoshkov wrote:
> "Vitaly Lipatov"  wrote:
> > As I see we need use libncursesw as the main curses library,
> > and have old curses support for compatibility with some old
> > systems.
>
> Forgot to ask, what application are you trying to fix? Is it
> freely downloadable?

Far

configure: add ncursesw detecting [try 2]

[Dmitry Timoshkov]

"Vitaly Lipatov"  wrote:
> > Most likely the real fixes in your patch are changing CP_ACP to CP_UNIXCP
> > and getting rid of hardcoded vkkeyscan_table. Do you have an evidence that
> > ncurses is so much broken that it really can't handle UTF-8? Does it work
> > if you change the locale to a not UTF-8 one?
>
> Dmitry, as I already post in this thread, we have test program which does not 
> print correctly ever in UTF-8 locale.
> Usual ncurses works in 8-bit locales like koi8-r correctly, but we can't 
> output in utf8 locale.

Have you been able to see any cyrillic text in wineconsole with the curses
backend at all? Even after changing CP_ACP to CP_UNIXCP I don't see anything.
This is with a KOI8-R locale. We need to make this configuration work first.

configure: add ncursesw detecting [try 2]

[Anatoly Lyutin]

Dmitry Timoshkov wrote:
> "Vitaly Lipatov"  wrote:
> > > Most likely the real fixes in your patch are changing CP_ACP to
> >
> > CP_UNIXCP
> > > and getting rid of hardcoded vkkeyscan_table. Do you have an
> >
> > evidence that
> > > ncurses is so much broken that it really can't handle UTF-8? Does
> >
> > it work
> > > if you change the locale to a not UTF-8 one?
> >
> > Dmitry, as I already post in this thread, we have test program which 
> > does not print correctly ever in UTF-8 locale.
> > Usual ncurses works in 8-bit locales like koi8-r correctly, but we 
> > can't output in utf8 locale.
>
> Have you been able to see any cyrillic text in wineconsole with the 
> curses
> backend at all? Even after changing CP_ACP to CP_UNIXCP I don't see 
> anything.
> This is with a KOI8-R locale. We need to make this configuration work 
> first.

Hello!
With This patch you will be able to see cyrillic symbols in wineconsole.

Changelog:
Add convertig from multibyte to widechar for correctly assignment in 
FillSimpleChar function.
Have Removed check in Refresh function : if(ch >127...), because 
cyrillic symbols has code more that 127.

configure: add ncursesw detecting [try 2]

[Dmitry Timoshkov]

"Anatoly Lyutin"  wrote
> With This patch you will be able to see cyrillic symbols in wineconsole.

Please send it to wine-patches. And obviously once your patch is committed
both places it fixes will need another fix to support multibyte encodings.