Transl: Getting rid of the conf directory (Part 1)

[Paul Vriens]

Hi,

In an effort to make conf more dynamic (and less depending on hardcoded
configuration) I'd like to get rid of the conf directory.

Let's make a first distinction here. We currently have a conf directory
used by the scripts/* files and one by the php/* files.

I'd like to discuss the scripts/conf directory first. Once that is done 
we can start discussing the php/conf directory

The scripts conf directory
==========================
The conf directory shows all the languages currently known by transl to 
be in use for translations. (We get a warning if a language is found in 
the source and we don't have a conf file for it.)

The most prominent use of the conf directory is to create a list of 
languages that will show up on the index page.
The conf file could hold a parameter "[ignore-sublang]". This parameter 
is used to change a LANG_XXX, SUBLANG_* into LANG_XXX, SUBLANG_NEUTRAL.

So, if we want to get rid of the hardcoded conf directory we need some 
logic that replaces the above.

The logic should hold the following rules:

1. LANG_XXX, SUBLANG_XXX_YYY is the highest level possible so we don't 
have to do anything here.

2. LANG_XXX, SUBLANG_DEFAULT (only 1 sublanguage in our code) should 
show LANG_XXX, SUBLANG LANG_XXX_YYY.

3. LANG_XXX, SUBLANG_DEFAULT (multiple sublanguages in our code) should 
show all possible LANG_XXX, SUBLANG_XXX_YYY.

4. LANG_XXX, SUBLANG_NEUTRAL (only 1 sublangugage in our code) should 
show LANG_XXX, SUBLANG_XXX_YYY.

5. LANG_XXX, SUBLANG_NEUTRAL (multiple sublanguages in our code) should 
mean show all possible LANG_XXX, SUBLANG_XXX_YYY.

(This logic is already somehow in place when the Portuguese issue was 
solved.)

I propose the following changes:

1. The first thing that should happen before any parsing is done is to 
get a list of all the different languages Wine is translated in.
2. Automatically create a scripts/conf directory that is based on the 
list found in step 1. This is temporary until the php part is converted 
as well. These conf files don't need the [hide] parameter or the 
[ignore-sublang] parameter anymore.
3. Change some of the php scripts to get rid of the two parameters 
mentioned in step 2. This basically means that we are not going to show 
pages anymore for the neutral languages (English is the exception).

Note for step 2.
The php scripts rely on the php/conf (copy of scripts/conf) directory 
for retrieving the name of a language. The kernel32 resources also have 
these names albeit sometimes a little different. I've noted the 
following differences:

conf-dir		kernel32
========		========
Chinese (Simplified)	Chinese (PRC)
Chinese (Traditional)	Chinese (Taiwan)
English (US)		English (United States)
English (Neutral)	English
Portugese (Brazilian)	Portuguese (Brazil)
Portugese (Portugal)	Portuguese (Portugal)

Do we want to go with the kernel32 names or should we have some kind of 
name override?

To prove my point I just looked at the list of translations we have, 
created the necessary files in scripts/conf, got rid of the 2 parameters 
and ran the Master.sh script to regenerate everything. The results were 
as expected and looked basically the same as on our transl page. After 
changing the index page to not use the [hide] parameter anymore but some 
other logic it looked exactly the same.

Thoughts, ideas, suggestions, questions on all of this?

(As said he php/conf directory can/will be discussed in a second email 
once this is "solved").

[Paul Vriens]

Paul Vriens wrote:
> Hi,
>
> In an effort to make conf more dynamic (and less depending on hardcoded

This should of course be "make transl more dynamic".

[Alexandre Julliard]

Paul Vriens  writes:
> - LANG_XXX, SUBLANG_DEFAULT (only 1 sublanguage in our code) should
>
> show LANG_XXX, SUBLANG LANG_XXX_YYY.
> - LANG_XXX, SUBLANG_DEFAULT (multiple sublanguages in our code)
>
> should show all possible LANG_XXX, SUBLANG_XXX_YYY.

There's nothing special about SUBLANG_DEFAULT, it's just the first
sublanguage.
> I propose the following changes:
> - The first thing that should happen before any parsing is done is to
>
> get a list of all the different languages Wine is translated in.

That should be a result of the parsing, I don't see why you'd need to do
this first.

[Paul Vriens]

Alexandre Julliard wrote:
> Paul Vriens  writes:
> > - LANG_XXX, SUBLANG_DEFAULT (only 1 sublanguage in our code) should
> >
> > show LANG_XXX, SUBLANG LANG_XXX_YYY.
> > - LANG_XXX, SUBLANG_DEFAULT (multiple sublanguages in our code)
> >
> > should show all possible LANG_XXX, SUBLANG_XXX_YYY.
>
> There's nothing special about SUBLANG_DEFAULT, it's just the first
> sublanguage.

Yeah, and it's not needed as you say. The only cases we need to cover 
are the SUBLANG_NEUTRAL ones that are converted to sublanguages if we 
have some.

The current code made that a mood argument as well as in ver.pl we only 
cover the SUBLANG_NEUTRAL cases as special ones and as said just 
creating some files in scripts/conf made it work.
> > I propose the following changes:
> > - The first thing that should happen before any parsing is done is to
> >
> > get a list of all the different languages Wine is translated in.
>
> That should be a result of the parsing, I don't see why you'd need to do
> this first.

Well, I was trying not to be too invasive here. Currently we do:

1. Go through the makefiles
     a. run "wrc --verify-translation
     b. run ver.pl

Step 1b currently needs the complete list of translations in the source 
to do it's work properly.

So maybe we should change that approach into:

1. Go through the makefiles
     a. run "wrc --verify-translation"
     b. create an hash/array/whatever of translations in the source
2. Loop through the results of the "wrc --verify-translation" runs.
     a. run ver.pl

We of course need that same list of translations in the php-scripts. The 
good things is that already have that in the form of files in the 
php/langs directory (created by ver.pl).

[Paul Vriens]

Paul Vriens wrote:
> Alexandre Julliard wrote:
> > Paul Vriens  writes:
> > > - LANG_XXX, SUBLANG_DEFAULT (only 1 sublanguage in our code) should
> > >
> > > show LANG_XXX, SUBLANG LANG_XXX_YYY.
> > > - LANG_XXX, SUBLANG_DEFAULT (multiple sublanguages in our code)
> > >
> > > should show all possible LANG_XXX, SUBLANG_XXX_YYY.
> >
> > There's nothing special about SUBLANG_DEFAULT, it's just the first
> > sublanguage.
>
> Yeah, and it's not needed as you say. The only cases we need to cover 
> are the SUBLANG_NEUTRAL ones that are converted to sublanguages if we 
> have some.
>
> The current code made that a mood argument as well as in ver.pl we only 
> cover the SUBLANG_NEUTRAL cases as special ones and as said just 
> creating some files in scripts/conf made it work.
> > > I propose the following changes:
> > > - The first thing that should happen before any parsing is done is to
> > >
> > > get a list of all the different languages Wine is translated in.
> >
> > That should be a result of the parsing, I don't see why you'd need to do
> > this first.
>
> Well, I was trying not to be too invasive here. Currently we do:
> - Go through the makefiles
>  a. run "wrc --verify-translation
>  b. run ver.pl
>
> Step 1b currently needs the complete list of translations in the source 
> to do it's work properly.
>
> So maybe we should change that approach into:
> - Go through the makefiles
>  a. run "wrc --verify-translation"
>  b. create an hash/array/whatever of translations in the source
> - Loop through the results of the "wrc --verify-translation" runs.
>  a. run ver.pl
>
> We of course need that same list of translations in the php-scripts. The 
> good things is that already have that in the form of files in the 
> php/langs directory (created by ver.pl).

Something like the attached splits the loop. We now need to pass that 
array (string) to ver.pl.

Or do you have some other idea's how this could/should be solved.

[Alexandre Julliard]

Paul Vriens  writes:
> Something like the attached splits the loop. We now need to pass that
> array (string) to ver.pl.
>
> Or do you have some other idea's how this could/should be solved.

I don't think we want to add even more information to pass between
scripts, there is already too much of it IMO. We should probably try to
merge scripts instead.

Also I'm not sure about skipping of the kernel32 locale files; it seems
to me we could just as well use that as the reference list, and display
every known language and sublanguage. At least it would make the index
page much more impressive ;-)

[Paul Vriens]

Alexandre Julliard wrote:
> Paul Vriens  writes:
> > Something like the attached splits the loop. We now need to pass that
> > array (string) to ver.pl.
> >
> > Or do you have some other idea's how this could/should be solved.
>
> I don't think we want to add even more information to pass between
> scripts, there is already too much of it IMO. We should probably try to
> merge scripts instead.
>
> Also I'm not sure about skipping of the kernel32 locale files; it seems
> to me we could just as well use that as the reference list, and display
> every known language and sublanguage. At least it would make the index
> page much more impressive ;-)

I actually was thinking the same about both items.

The downside of using the locale stuff is of course that our overall 
translations percentage goes down the drain ;) The upside is that we 
only need to do one --verify-translation run on kernel32 to get our list 
of possible translations.

I will have a look at merging checkmakefile.pl with ver.pl (already did 
that while testing).

[Paul Vriens]

Paul Vriens wrote:
> Alexandre Julliard wrote:
> > Paul Vriens  writes:
> > > Something like the attached splits the loop. We now need to pass that
> > > array (string) to ver.pl.
> > >
> > > Or do you have some other idea's how this could/should be solved.
> >
> > I don't think we want to add even more information to pass between
> > scripts, there is already too much of it IMO. We should probably try to
> > merge scripts instead.
> >
> > Also I'm not sure about skipping of the kernel32 locale files; it seems
> > to me we could just as well use that as the reference list, and display
> > every known language and sublanguage. At least it would make the index
> > page much more impressive ;-)
>
> I actually was thinking the same about both items.
>
> The downside of using the locale stuff is of course that our overall 
> translations percentage goes down the drain ;) The upside is that we 
> only need to do one --verify-translation run on kernel32 to get our list 
> of possible translations.
>
> I will have a look at merging checkmakefile.pl with ver.pl (already did 
> that while testing).

I played around a bit and some of the results are shown on the attached
screenshots.

Is this what you mean? The index page will (in the end) not show any 
neutral languages anymore.

We probably need some nicer sorting, but that looks trivial. Another 
idea is to have English (United States) as some kind of header instead 
of adding it to the table? All other translations are after all compared 
to English (US).

[Paul Vriens]

Paul Vriens wrote:
> Paul Vriens wrote:
> > Alexandre Julliard wrote:
> > > Paul Vriens  writes:
> > > > Something like the attached splits the loop. We now need to pass that
> > > > array (string) to ver.pl.
> > > >
> > > > Or do you have some other idea's how this could/should be solved.
> > >
> > > I don't think we want to add even more information to pass between
> > > scripts, there is already too much of it IMO. We should probably try to
> > > merge scripts instead.
> > >
> > > Also I'm not sure about skipping of the kernel32 locale files; it seems
> > > to me we could just as well use that as the reference list, and display
> > > every known language and sublanguage. At least it would make the index
> > > page much more impressive ;-)
> >
> > I actually was thinking the same about both items.
> >
> > The downside of using the locale stuff is of course that our overall 
> > translations percentage goes down the drain ;) The upside is that we 
> > only need to do one --verify-translation run on kernel32 to get our 
> > list of possible translations.
> >
> > I will have a look at merging checkmakefile.pl with ver.pl (already 
> > did that while testing).
>
> I played around a bit and some of the results are shown on the attached
> screenshots.
>
> Is this what you mean? The index page will (in the end) not show any 
> neutral languages anymore.
>
> We probably need some nicer sorting, but that looks trivial. Another 
> idea is to have English (United States) as some kind of header instead 
> of adding it to the table? All other translations are after all compared 
> to English (US).

Please don't look at the columns/numbers, the graphs are correct.