Death to win9x?

Dan Kegel wrote:
> I'm all for removing stuff that nobody needs anymore.

Who said nobody needs Win9x anymore?
> I imagine there
> are still a few popular older apps that will force us to keep some old
> win9x behaviors around, though, so we probably won't be able
> to remove the 'emulate win98' choice from winecfg any time soon.

Probably half of the apps I bought during the last years work with
Win9X.  Just the other day I saw a mciSendCommand16 in a log.
Well, 16bit != win9x.  Is it "Death to 16bit" you meant?

What's the motivation behind removing stuff *now* suddenly?  Is it the
perceived effort needed to fix the tests to all green colour on
testbot's win9x machines?
Why was this topic not discussed recently at Wineconf?

It would be a very bad idea to remove win9x from testbot. We need
these old systems to understand what behaviour old apps expect.
And I'm very thankful to Greg for his testbot service.

Without it, we would know next to nothing about win9x.  The fact that
a random old app works in native XP (in compatibility mode) teaches us
nothing for Wine.  Unless we start to run winetest in compatibility mode!

I've already mentioned in the past the idea to run testbot sometimes with all
broken() disabled, to assess failures on newer systems.  broken() may
shadow errors in the tests and lull Wine developers into wrong beliefs.

Paul Vriens wrote:
> The little bits and pieces we have in our code should stay as we 
> probably need that to cope with old apps that expect a different behavior.

Actually, I've even been thinking about implementing the opposite
behaviour: Old APIs are mostly used by old apps, while newer apps
moved to newer ones (e.g. DirectSound, then mmdev).  Therefore it
would make a lot of sense to have old APIs (e.g. WINMM/MCI/VfW) mimic the
behaviour observable when these apps were developed: win9x! (win3.x?)

We don't even know whether MS' compatibility mode does precisely that.

Regards,
    Jörg Höhle

Dan Kegel wrote:
> Like you, I'm afraid that dropping the win9x tests will lead
> to regressions old older apps.

Greg Geldorp wrote:
> However, they [Win9x VMs] would be available for selection
> if you click "Show all VMs" button when submitting a job.

This sounds very desirable.

However, this only allows to run one test at a time, i.e. the big picture is missing.
Therefore I'd suggest running winetest.exe on all testbot machines once a week
or some such, e.g. only for releases.

Later a student may write a thesis about the degradation of SW over time ;-)

An independent idea is to run a special winetest.exe compiled with
int broken(int condition) { return 0; }
once a week on (some of?) the regular machines.
However this is best performed *not* on release days, based on the
assumption that more people submit test data for release versions than
the day to day ones, and we don't want to "pollute" these with stricter tests.

Regards,
 Jörg Höhle

[Alexandre Julliard]

 writes:
> However, this only allows to run one test at a time, i.e. the big picture is missing.
> Therefore I'd suggest running winetest.exe on all testbot machines once a week
> or some such, e.g. only for releases.

That's not useful. The whole point is that we don't want to spend the
effort required to keep the tests error-free on platforms that we don't
care about. That makes it easier to write tests for platforms that
actually matter, which is a more productive use of everybody's time.

[Jerome Leclanche]

My very tentative $.02:
If there are tests we care about on Win9x (for old apps), wouldn't it
be easier to move them to their own file/folder, and only run those
and only those on 9x?

If the regular tests are going to lose all their broken() calls and
become meaningless on 9x, we still will want to test specific features
on old versions of Windows (for old apps as Dan mentioned). That way,
tests are still easier to write, but we don't lose that ability.

JL

On Thu, Dec 2, 2010 at 1:30 PM, Alexandre Julliard  wrote:
>  writes:
> > However, this only allows to run one test at a time, i.e. the big picture is missing.
> > Therefore I'd suggest running winetest.exe on all testbot machines once a week
> > or some such, e.g. only for releases.
>
> That's not useful. The whole point is that we don't want to spend the
> effort required to keep the tests error-free on platforms that we don't
> care about. That makes it easier to write tests for platforms that
> actually matter, which is a more productive use of everybody's time.
>
> --
> Alexandre Julliard
> [e-mail]

[André Hentschel]

Am 02.12.2010 13:46, schrieb [e-mail]:
> An independent idea is to run a special winetest.exe compiled with
> int broken(int condition) { return 0; }

I recently sent a patch to have a new debuglevel "2", but AJ wisely pointed out we can simply set WINETEST_PLATFORM=wine on Windows which has the same effect.
So no need for such hacks :D

André Hentschel wrote:
> I recently sent a patch to have a new debuglevel "2", but AJ wisely
> pointed out we can simply set WINETEST_PLATFORM=wine on Windows
> which has the same effect. So no need for such hacks :D

I once submitted a similar command line patch: runtest --strict :-)

AJ is right.  The only problem is that testbot does not allow me to set an environment
variable. All testbot knows are command line arguments.

One idea would be to be able to run winetest.exe inside testbot with enough
command line arguments that it runs all tests and submits results on its own.

Regards,
 Jörg Höhle

[André Hentschel]

Am 02.12.2010 19:17, schrieb [e-mail]:
> André Hentschel wrote:
> > I recently sent a patch to have a new debuglevel "2", but AJ wisely
> > pointed out we can simply set WINETEST_PLATFORM=wine on Windows
> > which has the same effect. So no need for such hacks :D
>
> I once submitted a similar command line patch: runtest --strict :-)
>
> AJ is right.  The only problem is that testbot does not allow me to set an environment
> variable. All testbot knows are command line arguments.

you are free to send a patch for testbot to handle that (yes, testbot is open source) :)

Hi,

André Hentschel wrote:
> actually we just always need to do something like skip() or broken()
> and that's nothing else as ignoring the test results of 9x

Well, I could sympathize with the idea of removing all broken(/*win9x*/)
and leave only skip() such that the tests don't crash prematurely.

Does test.winehq.org refuse win9x test data since 2010-12-17?
This is not a good idea at all.
We had a couple of real machines that were submitting results, e.g. Saulius's machines.
Even when test failures on win9x are not a criteria for rejecting patches, that
must not mean that we become blind on what happens on the win9x side.

What seems to happen now is overshooting.
 + Don't let testbot run patches against win9x anymore -- ok
    which is quite different from ignoring win9x test failures.
 + Still have them alive for developers     -- great
 - Drop win9x data from test.winehq.org -- WTF?

People are still using many apps written in the win9x era and
not all of those have platinum rating. We still need to know how
those old systems behaved differently.

Regards,
 Jörg Höhle

[Jacek Caban]

On 12/22/10 5:52 PM, [e-mail] wrote:
> Hi,
>
> André Hentschel wrote:
> > actually we just always need to do something like skip() or broken()
> > and that's nothing else as ignoring the test results of 9x
>
> Well, I could sympathize with the idea of removing all broken(/*win9x*/)
> and leave only skip() such that the tests don't crash prematurely.
>
> Does test.winehq.org refuse win9x test data since 2010-12-17?

Yes, see my patch:
http://www.winehq.org/pipermail/wine-patches/2010-December/096908.html
> This is not a good idea at all.
> We had a couple of real machines that were submitting results, e.g. Saulius's machines.
> Even when test failures on win9x are not a criteria for rejecting patches, that
> must not mean that we become blind on what happens on the win9x side.

It was said already a few times: dropping win9x tests has nothing to do 
with dropping win9x support. These test results weren't helpful nor 
useful anyways.

Jacek

[Saulius Krasuckas]

* On Wed, 22 Dec 2010, Jacek Caban wrote:
> dropping win9x tests has nothing to do with dropping win9x support.

It has, but in a small degree -- if win9x support regresses now, these 
existing few cases of testing win9x specifics won't do their job.
> These test results weren't helpful nor useful anyways.

And how is the project going to get rid of win9x specifics in the 
winetest?  Some of them were being skip()ed, some broken() and some of 
them were left just as valid (eg. ERROR_NOT_LOGGED_ON).

You will you let them rot?

But I see the pluses too.  Now if I stuck upon some win9x-specifics, I can 
change WinAPI function without even needing a test case.  Because it won't 
be tested anyway, right? :)

I guess more patches are going to go through since<g>.

S.

[Wolfram Sang]

> The only problem is that testbot does not allow me to set an environment
> variable. All testbot knows are command line arguments.

Well, you can use 7zip and its SFX-capabilities to do a lot more with 
testbot. I used this to invoke winhlp showing a specific helpfile :)
For your case, you might embed a shell script or similar?

Regards,

Wolfram