Hi WINE-devel list (cross posted to Linux-IL),
After a lively discussion in the Israeli Linux users mailing list, it appears that the best solution will be to give a command line (or config) option to use an external library (fribidi) for the RTL rendering. I would still implement this internally, as the extrenal library is more Unicode and less Windows complient, but that's besides the point.
Adding a ./configure option to do so would probably be quite feasable (I don't know how to do it at the moment, but I am sure that will not be the disqualifying factor), but I have been quite hesitant to go that way until now, for the following reasons: A. Fribidi will have to be present during building WINE. That, in itself, is not a majour obstacle, but B. Having Fribidi present during compile will add Fribidi to the WINE dependancies.
I am afraid that the consequence of B will be that no packager in his right mind will use that option. As that works in direct contradiction to the reason I entered the WINE project to begin with, I am trying to opt for a better solution. I'm seeing two possible solutions to this problem: A. If Fribidi was present during compile, check for its existance during run time. If not present, don't enable the run time option. or B. Copy (port?) Fribidi into the WINE code. It's LGPL, so the license does allow that.
The consequence of B is that I will need to keep an eye on Fribidi, and keep it up to date whenever something changes there. I am very reluctant to do so. I would much rather go for the "A" solution. The problem is that I know how to do this in Windows, but not in Linux. Any help anyone can offer me?
Shachar
Hi,
On Tue, 18 Jun 2002, Shachar Shemesh wrote: [snip]
A. If Fribidi was present during compile, check for its existance during run time. If not present, don't enable the run time option. or B. Copy (port?) Fribidi into the WINE code. It's LGPL, so the license does allow that.
C. If Fribidi was present during compile, link it statically, so that there will be no dependency on fribidi for WINE package.
The consequence of B and C is that WINE will not use the latest fribidi on machine.
But that may be right, since I wonder if you are going to convert UTF32<->UTF16 all way. You need a 16bit fribidi, means a somehow self-taylored version, so go with B and install a 16bit fribidi on your compile machine, and build WINE. No license problem, and I will let you know of fribidi new releases ;-).
The consequence of B is that I will need to keep an eye on Fribidi, and keep it up to date whenever something changes there. I am very reluctant to do so. I would much rather go for the "A" solution. The problem is that I know how to do this in Windows, but not in Linux. Any help anyone can offer me? Shachar
On Tue, 18 Jun 2002, Shachar Shemesh wrote: [...]
Adding a ./configure option to do so would probably be quite feasable (I don't know how to do it at the moment, but I am sure that will not be the disqualifying factor), but I have been quite hesitant to go that way until now, for the following reasons: A. Fribidi will have to be present during building WINE. That, in itself, is not a majour obstacle, but B. Having Fribidi present during compile will add Fribidi to the WINE dependancies.
A way to deal with B is to do like for FreeType, i.e. load the library dynamically at runtime, and disable support for it if that fails.
But I have another question which you have probably already envisioned (might even have already discussed on the list though I can't remember right now): FriBiDi being a Unix Unicode library, I assume it uses 4-byte Unicode characters? Won't it be a problem since Wine uses 2-byte Unicode characters?
That may be another argument for integrating FriBiDi with Wine, or maybe: * adding a compile option so that FriBiDi works with either 2 or 4 byte characters * make it possible to write a configure check to detect whether the FriBiDi library is compiled for 2 or 4 byte characters * then packagers would compile Wine with this special FriBiDi library, and ship it with Wine
How does that sound?
-- Francois Gouget fgouget@free.fr http://fgouget.free.fr/ May your Tongue stick to the Roof of your Mouth with the Force of a Thousand Caramels.
On Tue, 18 Jun 2002, Francois Gouget wrote:
On Tue, 18 Jun 2002, Shachar Shemesh wrote: A way to deal with B is to do like for FreeType, i.e. load the library dynamically at runtime, and disable support for it if that fails.
But I have another question which you have probably already envisioned (might even have already discussed on the list though I can't remember right now): FriBiDi being a Unix Unicode library, I assume it uses 4-byte Unicode characters? Won't it be a problem since Wine uses 2-byte Unicode characters?
That may be another argument for integrating FriBiDi with Wine, or maybe:
Yes, this is what I suggested too.
- adding a compile option so that FriBiDi works with either 2 or 4 byte characters
I'm working on it, currently there is no compile time option, but you can do it by changing one character from 4 to 2 in a header file, and it will regenerate the tables.
- make it possible to write a configure check to detect whether the FriBiDi library is compiled for 2 or 4 byte characters
It's already possible, something like this:
AC_TRY_RUN ([ #include <fribidi/fribidi_types.h> int main() { return sizeof (FriBidiChar == 2) ? 0 : 1; } ], FRIBIDI16=true, FRIBIDI16=false )
- then packagers would compile Wine with this special FriBiDi library, and ship it with Wine
How does that sound?
The right choice.
-- Francois Gouget fgouget@free.fr http://fgouget.free.fr/ May your Tongue stick to the Roof of your Mouth with the Force of a Thousand Caramels.
On Tue, Jun 18, 2002 at 07:32:58PM +0300, Shachar Shemesh wrote:
I am afraid that the consequence of B will be that no packager in his right mind will use that option. As that works in direct contradiction to the reason I entered the WINE project to begin with, I am trying to opt for a better solution. I'm seeing two possible solutions to this problem: A. If Fribidi was present during compile, check for its existance during run time. If not present, don't enable the run time option. or B. Copy (port?) Fribidi into the WINE code. It's LGPL, so the license does allow that.
The consequence of B is that I will need to keep an eye on Fribidi, and keep it up to date whenever something changes there. I am very reluctant to do so. I would much rather go for the "A" solution. The problem is that I know how to do this in Windows, but not in Linux. Any help anyone can offer me?
Have a look at dlls/gdi/freetype.c where we use wine_dlopen and friends to do this with libfreetype.so
Huw.
Shachar Shemesh linuxil@consumer.org.il writes:
I am afraid that the consequence of B will be that no packager in his right mind will use that option. As that works in direct contradiction to the reason I entered the WINE project to begin with, I am trying to opt for a better solution. I'm seeing two possible solutions to this problem:
A. If Fribidi was present during compile, check for its existance during run time. If not present, don't enable the run time option.
or B. Copy (port?) Fribidi into the WINE code. It's LGPL, so the license does allow that.
I'd strongly suggest doing B. We need 16-bit Unicode chars anyway so you probably can't use the standard Fribidi. And this way we can share the Unicode tables with the rest of our code instead of duplicating a whole lot of it. All you really need from Fribidi is the algorithms, and these are not going to change often anyway so copying them into Wine shouldn't be too bad.
On 18 Jun 2002, Alexandre Julliard wrote:
Shachar Shemesh linuxil@consumer.org.il writes:
I am afraid that the consequence of B will be that no packager in his right mind will use that option. As that works in direct contradiction to the reason I entered the WINE project to begin with, I am trying to opt for a better solution. I'm seeing two possible solutions to this problem:
A. If Fribidi was present during compile, check for its existance during run time. If not present, don't enable the run time option.
or B. Copy (port?) Fribidi into the WINE code. It's LGPL, so the license does allow that.
I'd strongly suggest doing B. We need 16-bit Unicode chars anyway so you probably can't use the standard Fribidi. And this way we can share the Unicode tables with the rest of our code instead of duplicating a whole lot of it. All you really need from Fribidi is the algorithms, and these are not going to change often anyway so copying them into Wine shouldn't be too bad.
Your are quite right, Fribidi's main algorithms are not going to change anymore, the things I'm playing with these days are all interface and other side stuff (BTW if you want arabic joining too, things become a little different). The fribidi unicode tables (for just BMP = 16bit) takes something between 2k and 8k (space vs. speed configurable), so don't worry about that too.
Behdad Esfahbod wrote:
Your are quite right, Fribidi's main algorithms are not going to change anymore, the things I'm playing with these days are all interface and other side stuff (BTW if you want arabic joining too, things become a little different).
I will, eventually. Then again, I don't think that integrating a first version that isn't perfect in that respect, and then doing the port again when you guys are done, is such a bad idea.
The fribidi unicode tables (for just BMP = 16bit) takes something between 2k and 8k (space vs. speed configurable), so don't worry about that too.
Problem is that WINE already has its own unicode tables, and they do not hold enough room for the the 3.0 bidi algo (havn't checked what FriBidi is doing yet). I think it will be better for WINE (but not for my free time) to merge the information (probably very similar already).
As for compiling Fribidi with UTF-16 - from what I understood from what you said before, such a thing may cause reordering problems if Fribidi decides, for whatever reason, that a surrogate character needs to be right to left. I am not 100% familiar with the bidi algorythm yet, but won't marking all surrogate forms (both slices) as strong left to right solve this problem?
If not, we can always put in special handling that reorders surrogates as a pair.
Shachar
On Wed, 19 Jun 2002, Shachar Shemesh wrote:
Behdad Esfahbod wrote:
Your are quite right, Fribidi's main algorithms are not going to change anymore, the things I'm playing with these days are all interface and other side stuff (BTW if you want arabic joining too, things become a little different).
I will, eventually. Then again, I don't think that integrating a first version that isn't perfect in that respect, and then doing the port again when you guys are done, is such a bad idea.
Yes, Ofcourse.
The fribidi unicode tables (for just BMP = 16bit) takes something between 2k and 8k (space vs. speed configurable), so don't worry about that too.
Problem is that WINE already has its own unicode tables, and they do not hold enough room for the the 3.0 bidi algo (havn't checked what FriBidi is doing yet). I think it will be better for WINE (but not for my free time) to merge the information (probably very similar already).
I wonder if WINE already has bidi tables of unicode too, if it has, no problem using it.
As for compiling Fribidi with UTF-16 - from what I understood from what you said before, such a thing may cause reordering problems if Fribidi decides, for whatever reason, that a surrogate character needs to be right to left. I am not 100% familiar with the bidi algorythm yet, but won't marking all surrogate forms (both slices) as strong left to right solve this problem?
The problem you are talking about will show itself when some one use RLO..PDF pairs (override embeddings) over some surrogate pairs, but this is not the main problem, I can hach fribidi to take care of surrogates and reorder them back if needed (really easy), the real problem is that when using UTF-16, fribidi will assume all surrogate characters LTR (strong left to right), but there *are* non-LTR characters there already (like language tags), and it may cause to different renderings. If you want conformance, UTF-32 is needed ;-(.
If not, we can always put in special handling that reorders surrogates as a pair.
Shachar
Behdad Esfahbod wrote:
I wonder if WINE already has bidi tables of unicode too, if it has, no problem using it.
Wrong, I'm afraid. The WINE BiDi table only has the information as defined in Window's GetStringType function. Quoting: Name Value Meaning *Strong*
C2_LEFTTORIGHT 0x0001 Left to right C2_RIGHTTOLEFT 0x0002 Right to left *Weak*
C2_EUROPENUMBER 0x0003 European number, European digit C2_EUROPESEPARATOR 0x0004 European numeric separator C2_EUROPETERMINATOR 0x0005 European numeric terminator C2_ARABICNUMBER 0x0006 Arabic number C2_COMMONSEPARATOR 0x0007 Common numeric separator *Neutral*
C2_BLOCKSEPARATOR 0x0008 Block separator C2_SEGMENTSEPARATOR 0x0009 Segment separator C2_WHITESPACE 0x000A White space C2_OTHERNEUTRAL 0x000B Other neutrals *Not applicable*
C2_NOTAPPLICABLE 0x0000 No implicit directionality (for example, control codes)
If that's enough - great. I was under the impression this is not enough for 3.0 Unicode implementation.
As for compiling Fribidi with UTF-16 - from what I understood from what you said before, such a thing may cause reordering problems if Fribidi decides, for whatever reason, that a surrogate character needs to be right to left. I am not 100% familiar with the bidi algorythm yet, but won't marking all surrogate forms (both slices) as strong left to right solve this problem?
The problem you are talking about will show itself when some one use RLO..PDF pairs (override embeddings) over some surrogate pairs, but this is not the main problem, I can hach fribidi to take care of surrogates and reorder them back if needed (really easy), the real problem is that when using UTF-16, fribidi will assume all surrogate characters LTR (strong left to right), but there *are* non-LTR characters there already (like language tags), and it may cause to different renderings. If you want conformance, UTF-32 is needed ;-(.
Hmm. Ok, I'll write a mental note to look at it later.
If not, we can always put in special handling that reorders surrogates as a pair.
That sounds like the ultimate solution? If we already implement re-reordering for wrongly ordered surrogates, we don't have to mark them all as strong LTR. Right?
Shachar
On Wed, 19 Jun 2002, Shachar Shemesh wrote:
Behdad Esfahbod wrote:
I wonder if WINE already has bidi tables of unicode too, if it has, no problem using it.
Wrong, I'm afraid. The WINE BiDi table only has the information as defined in Window's GetStringType function. Quoting: Name Value Meaning *Strong*
C2_LEFTTORIGHT 0x0001 Left to right C2_RIGHTTOLEFT 0x0002 Right to left *Weak*
C2_EUROPENUMBER 0x0003 European number, European digit C2_EUROPESEPARATOR 0x0004 European numeric separator C2_EUROPETERMINATOR 0x0005 European numeric terminator C2_ARABICNUMBER 0x0006 Arabic number C2_COMMONSEPARATOR 0x0007 Common numeric separator *Neutral*
C2_BLOCKSEPARATOR 0x0008 Block separator C2_SEGMENTSEPARATOR 0x0009 Segment separator C2_WHITESPACE 0x000A White space C2_OTHERNEUTRAL 0x000B Other neutrals *Not applicable*
C2_NOTAPPLICABLE 0x0000 No implicit directionality (for example, control codes)
If that's enough - great. I was under the impression this is not enough for 3.0 Unicode implementation.
You can have a look at 3.0 types at fribidi_types.h, they are 19, which 5 are explicit bidi marks, so there should still be something missing, but no problem to use them, really easy for fribidi.
As for compiling Fribidi with UTF-16 - from what I understood from what you said before, such a thing may cause reordering problems if Fribidi decides, for whatever reason, that a surrogate character needs to be right to left. I am not 100% familiar with the bidi algorythm yet, but won't marking all surrogate forms (both slices) as strong left to right solve this problem?
The problem you are talking about will show itself when some one use RLO..PDF pairs (override embeddings) over some surrogate pairs, but this is not the main problem, I can hach fribidi to take care of surrogates and reorder them back if needed (really easy), the real problem is that when using UTF-16, fribidi will assume all surrogate characters LTR (strong left to right), but there *are* non-LTR characters there already (like language tags), and it may cause to different renderings. If you want conformance, UTF-32 is needed ;-(.
Hmm. Ok, I'll write a mental note to look at it later.
If not, we can always put in special handling that reorders surrogates as a pair.
That sounds like the ultimate solution? If we already implement re-reordering for wrongly ordered surrogates, we don't have to mark them all as strong LTR. Right?
Ok, we will do this, but if not LTR, what other type? LTR is the default type for all undefined chars and so, and really the best huristic...
Shachar
-
Alexandre Julliard -
Behdad Esfahbod -
Francois Gouget -
Huw D M Davies -
Shachar Shemesh