vartest.c - major pain in the build process

[Krzysztof Foltman]

Is there any reason for gcc 3.3.1 to spend several minutes on compiling 
dlls/oleaut32/tests/vartest.c ?

I guess it can be explained by optimization stage going nuts on 
compiling long functions with many branches (contained in the macros), 
but there must be a way to speed this up.

It's barely tolerable now, and upgrading gcc is not an option for everyone.

Krzysztof

[Dmitry Timoshkov]

"Krzysztof Foltman"  wrote:
> Is there any reason for gcc 3.3.1 to spend several minutes on compiling 
> dlls/oleaut32/tests/vartest.c ?
>
> I guess it can be explained by optimization stage going nuts on 
> compiling long functions with many branches (contained in the macros), 
> but there must be a way to speed this up.
>
> It's barely tolerable now, and upgrading gcc is not an option for everyone.

Well, I'd argue that compilation time of full Wine tree is much longer
than that single test. If you compile from source it's your choice.

I wonder why you don't complain about programs/winetest linking times.

Even more drastic route to go is to go and complain to gcc people,
that's certainly not a Wine fault.

[Marcus Meissner]

On Sat, Feb 26, 2005 at 10:32:11PM +0800, Dmitry Timoshkov wrote:
> "Krzysztof Foltman"  wrote:
> > Is there any reason for gcc 3.3.1 to spend several minutes on compiling 
> > dlls/oleaut32/tests/vartest.c ?
> >
> > I guess it can be explained by optimization stage going nuts on 
> > compiling long functions with many branches (contained in the macros), 
> > but there must be a way to speed this up.
> >
> > It's barely tolerable now, and upgrading gcc is not an option for everyone.
>
> Well, I'd argue that compilation time of full Wine tree is much longer
> than that single test. If you compile from source it's your choice.
>
> I wonder why you don't complain about programs/winetest linking times.
>
> Even more drastic route to go is to go and complain to gcc people,
> that's certainly not a Wine fault.

well, 25 minutes for a single file and 25 minutes for the rest of the tree? :)

gcc4 fares better for this test already.

I personally change dlls/oleaut32/tests/Makefile to use -O1 after reconfigure.

Ciao, Marcus

[Mike Hearn]

On Sat, 26 Feb 2005 15:56:56 +0100, Marcus Meissner wrote:
> well, 25 minutes for a single file and 25 minutes for the rest of the tree? :)

It doesn't take 25 minutes here, how much memory do you have? It takes
more like 5, which is still excessive :(

I disable winetest in my tree entirely, and have done for a long time. It
should be disabled in CVS as well, nobody should be building winetest on
Linux, it just encourages people to submit useless results by saying "oooh
I wonder what happens if I do this". Worse this isn't just about it taking
too long, on my machine attempting to compile winetest actually crushes
it totally. It descends into swap hell and never seems to re-emerge.

I don't think having to hack the build system to avoid a full system hang
is really a good thing, even if it is the kernels fault.

thanks -mike

[Jakob Eriksson]

Mike Hearn wrote:
> I disable winetest in my tree entirely, and have done for a long time. It
> should be disabled in CVS as well, nobody should be building winetest on
> Linux, it just encourages people to submit useless results by saying "oooh
> I wonder what happens if I do this". Worse this isn't just about it taking

Actually there may be some value in running winetest on Linux. [*]

Also, that won't stop people from running winetest on linux, I'll bet
$5 winetest.exe downloads from WRT will keep showing up.

* - We not only get statistics of how the Windows API works. We also
get coverage of how well Wine is implemented in the wild, with different
versions of Linux.
> too long, on my machine attempting to compile winetest actually crushes
> it totally. It descends into swap hell and never seems to re-emerge.

That said, maybe winetest should not build by default. That
doesn't sound like a lot of fun.

regards,
Jakob

[Krzysztof Foltman]

Jakob Eriksson wrote:
> Also, that won't stop people from running winetest on linux, I'll bet
> $5 winetest.exe downloads from WRT will keep showing up.

Yes. But should something that's useless for the majority of users and 
developers be enabled by default ? Looks like significant cost with 
little benefit.
> That said, maybe winetest should not build by default. That
> doesn't sound like a lot of fun.

Sounds like a good idea. And, by the way, a configure switch to disable 
all the DirectX modules would be nice too. One doesn't need DirectX to 
run business software. Or to work on a non-multimedia control like 
RichEdit20 ;) Or, maybe, a switch to skip building particular DLLs.

Krzysztof

[Dmitry Timoshkov]

"Jakob Eriksson"  wrote:
> Also, that won't stop people from running winetest on linux, I'll bet
> $5 winetest.exe downloads from WRT will keep showing up.

I'd actually rise the question of sending the results of running winetest
on an invisible desktop under Windows. For instance I'm interested to see
how my recently added user32 tests behave on different Windows platforms,
but the tests which currently present on http://test.winehq.org/data are
completely unusable. I strongly ask to stop posing tests running on an
invisible desktop and consider that tests broken, since almost all win32
APIs are influenced by the fact of running in that environment. Moreover,
Wine doesn't run in that mode at all, so we can't compare apples to apples
in that case. And the apps most of the developers/users care about should be
run on a visible desktop anyway.
> That said, maybe winetest should not build by default. That
> doesn't sound like a lot of fun.

Alexandre already stated many times that there is no point to have
anything in the CVS tree if it's not compiled by default. A not used
code tends to be rotten very fast.

[Mike Hearn]

On Sun, 27 Feb 2005 17:52:44 +0800, Dmitry Timoshkov wrote:
> Alexandre already stated many times that there is no point to have
> anything in the CVS tree if it's not compiled by default. A not used
> code tends to be rotten very fast.

Winetest is built though, as part of the nightly test winrash
infrastructure. There's no point building in stock developer/end user
installs but that doesn't mean it'd never be built at all.

thanks -mike

[Ferenc Wagner]

"Dmitry Timoshkov"  writes:
> I'd actually rise the question of sending the results of running winetest
> on an invisible desktop under Windows. For instance I'm interested to see
> how my recently added user32 tests behave on different Windows platforms,
> but the tests which currently present on http://test.winehq.org/data are
> completely unusable. I strongly ask to stop posing tests running on an
> invisible desktop and consider that tests broken, since almost all win32
> APIs are influenced by the fact of running in that environment. Moreover,
> Wine doesn't run in that mode at all, so we can't compare apples to apples
> in that case. And the apps most of the developers/users care about should be
> run on a visible desktop anyway.

I was under the impression that most of the tests are
independent of desktop visibility.  And the winrash service
is the only way to get several reports quickly for a new
test.  If it wasn't automatic, it would take a good while
until the tests are run.  So I wanted to keep the best of
both ways by introducing a new flag according to your
recommendation:

static int running_on_visible_desktop ()
{
    return (GetWindowLongA (GetDesktopWindow (), GWL_STYLE) & WS_VISIBLE) != 0;
}

Looks like it doesn't quite work, though: it returns 1 all
the time under Win9x and NT.  Can you perhaps tell why?

[Dmitry Timoshkov]

"Ferenc Wagner"  wrote:
> I was under the impression that most of the tests are
> independent of desktop visibility.

Not really. Any API which directly or indirectly creates windows or uses
GDI is affected by the desktop visibility. Also, as I already pointed out,
Wine doesn't run in that mode at all, so we can't compare apples to apples
in that case. And the apps most of the developers/users care about should be
run on a visible desktop anyway.
> And the winrash service
> is the only way to get several reports quickly for a new
> test.

Why? What prevents someone to run the tests manually in interactive mode
once a day? If that someone can't or won't do it, then we have to find
another someone. I'm pretty sure that there are enough not lazy people
wishing to help we could to choose from.
> If it wasn't automatic, it would take a good while
> until the tests are run.  So I wanted to keep the best of
> both ways by introducing a new flag according to your
> recommendation:
>
> static int running_on_visible_desktop ()
> {
>     return (GetWindowLongA (GetDesktopWindow (), GWL_STYLE) & WS_VISIBLE) != 0;
> }
>
> Looks like it doesn't quite work, though: it returns 1 all
> the time under Win9x and NT.  Can you perhaps tell why?

I don't know. And figuring out why it doesn't work is completely useless IMO,
since the results of the tests running on an invisible desktop can't be used
for a reasonable comparison.

[Ivan Leo Puoti]

Dmitry Timoshkov wrote:
> Why? What prevents someone to run the tests manually in interactive mode
> once a day? If that someone can't or won't do it, then we have to find
> another someone. I'm pretty sure that there are enough not lazy people
> wishing to help we could to choose from.

We can just have winrash run in interactive mode. Once tests are ready to run, a message pops up
saying "new tests available" or something of the sort, the user then chooses to run them now or
later, like the windows automatic updates. Like this we could test both visible and invisible 
desktops, and could have a fair amount of results for both environments.

Ivan.

[Dmitry Timoshkov]

"Ivan Leo Puoti"  wrote:
> We can just have winrash run in interactive mode. Once tests are ready to run, a message pops up
> saying "new tests available" or something of the sort, the user then chooses to run them now or
> later, like the windows automatic updates. Like this we could test both visible and invisible 
> desktops, and could have a fair amount of results for both environments.

I have to repeat again: Winword, Photoshop, Internet Explorer or any other
application I and most (if not all) developers and users are care about do not
run on an invisible desktop, therefore I don't care about tests running in that
mode at all. It's not simply an optional mode, that's a mode completely useless
for vast majority of Wine users/developers.

[Ivan Leo Puoti]

Dmitry Timoshkov wrote:
> "Ivan Leo Puoti"  wrote:
> > We can just have winrash run in interactive mode. Once tests are ready to run, a message pops up
> > saying "new tests available" or something of the sort, the user then chooses to run them now or
> > later, like the windows automatic updates. Like this we could test both visible and invisible 
> > desktops, and could have a fair amount of results for both environments.
>
> I have to repeat again: Winword, Photoshop, Internet Explorer or any other
> application I and most (if not all) developers and users are care about do not
> run on an invisible desktop, therefore I don't care about tests running in that
> mode at all. It's not simply an optional mode, that's a mode completely useless
> for vast majority of Wine users/developers.

OK, then we can have winrash only run them in interactive mode, end of 
story.

Ivan.

[Dimitrie O. Paun]

On Mon, Feb 28, 2005 at 01:10:25PM +0100, Ivan Leo Puoti wrote:
> OK, then we can have winrash only run them in interactive mode, end of 
> story.

Don't need to do that, there are plenty of tests that run fine
with an invisible desktop, there's enough value in having some
automated tests. Maybe we need to separate the results...

[Ivan Leo Puoti]

Dimitrie O. Paun wrote:
> Don't need to do that, there are plenty of tests that run fine
> with an invisible desktop, there's enough value in having some
> automated tests. Maybe we need to separate the results...

I don't really care at all, I just think winrash should be kept and 
should be improved
to run interactive tests too.

Ivan.

[Jakob Eriksson]

Dmitry Timoshkov wrote:
> "Ivan Leo Puoti"  wrote:
> > We can just have winrash run in interactive mode. Once tests are ready to run, a message pops up
> > saying "new tests available" or something of the sort, the user then chooses to run them now or
> > later, like the windows automatic updates. Like this we could test both visible and invisible 
> > desktops, and could have a fair amount of results for both environments.
>
> I have to repeat again: Winword, Photoshop, Internet Explorer or any other
> application I and most (if not all) developers and users are care about do not
> run on an invisible desktop, therefore I don't care about tests running in that
> mode at all. It's not simply an optional mode, that's a mode completely useless
> for vast majority of Wine users/developers.

For example file APIs many applications depend upon, and these
are totally unaffected by desktop visibility.

regards,
Jakob

[Dmitry Timoshkov]

"Jakob Eriksson"  wrote:
> > I have to repeat again: Winword, Photoshop, Internet Explorer or any other
> > application I and most (if not all) developers and users are care about do not
> > run on an invisible desktop, therefore I don't care about tests running in that
> > mode at all. It's not simply an optional mode, that's a mode completely useless
> > for vast majority of Wine users/developers.
>
> For example file APIs many applications depend upon, and these
> are totally unaffected by desktop visibility.

We can't really separate the tests, but still want the tests to show
reasonable results in order to compare behaviour with Wine and to track
the regressions. I don't see the point of publishing broken tests, and
why running tests interactively can't be made a mandatory requirement.

[Ivan Leo Puoti]

On Tue, 1 Mar 2005 08:29:00 +0800, Dmitry Timoshkov  wrote:
> "Jakob Eriksson"  wrote:
> > > I have to repeat again: Winword, Photoshop, Internet Explorer or any other
> > > application I and most (if not all) developers and users are care about do not
> > > run on an invisible desktop, therefore I don't care about tests running in that
> > > mode at all. It's not simply an optional mode, that's a mode completely useless
> > > for vast majority of Wine users/developers.
> >
> > For example file APIs many applications depend upon, and these
> > are totally unaffected by desktop visibility.
>
> We can't really separate the tests, but still want the tests to show
> reasonable results in order to compare behaviour with Wine and to track
> the regressions. I don't see the point of publishing broken tests, and
> why running tests interactively can't be made a mandatory requirement.

It certanly should be at least an option.

Ivan.

[Dmitry Timoshkov]

"Ivan Leo Puoti"  wrote:
> > We can't really separate the tests, but still want the tests to show
> > reasonable results in order to compare behaviour with Wine and to track
> > the regressions. I don't see the point of publishing broken tests, and
> > why running tests interactively can't be made a mandatory requirement.
>
> It certanly should be at least an option.

In order to see what tests are affected by desktop visibility and which don't
you have to run in both modes and compare the results. Why do it twice if it
can be avoided? Right now any failure in the tests can be attributed to the
desktop visibility, and until it's fixed nobody is going to fix the tests.

Taking into account recent drastic changes in window management and the fact
they are mainly based on the test results, I don't see how you are going
to fix regressions in Wine without seeing how other Windows versions behave,
not only XP flavour Alexandre has tested on.

[Ivan Leo Puoti]

Dmitry Timoshkov wrote:
> In order to see what tests are affected by desktop visibility and which don't
> you have to run in both modes and compare the results. Why do it twice if it
> can be avoided? Right now any failure in the tests can be attributed to the
> desktop visibility, and until it's fixed nobody is going to fix the tests.

You should email the guy who maintains winrash :-)

Ivan.

[Dmitry Timoshkov]

"Ivan Leo Puoti"  wrote:
> > In order to see what tests are affected by desktop visibility and which don't
> > you have to run in both modes and compare the results. Why do it twice if it
> > can be avoided? Right now any failure in the tests can be attributed to the
> > desktop visibility, and until it's fixed nobody is going to fix the tests.
>
> You should email the guy who maintains winrash :-)

First of all that should be a collective request, not just a single voice
in the dark. And I believe it should be a Wine test site maintainers
responsibility (it's WineHQ web space after all), winrash has no direct
relationship to Wine development since it's not in the Wine CVS.

[Ferenc Wagner]

"Dmitry Timoshkov"  writes:
> "Ivan Leo Puoti"  wrote:
> > > In order to see what tests are affected by desktop visibility and which don't
> > > you have to run in both modes and compare the results. Why do it twice if it
> > > can be avoided? Right now any failure in the tests can be attributed to the
> > > desktop visibility, and until it's fixed nobody is going to fix the tests.
> >
> > You should email the guy who maintains winrash :-)
>
> First of all that should be a collective request, not just
> a single voice in the dark.

Yours may be a single voice, but definitely not in the dark,
I hope.  Even if I'm a little dense sometimes.
> And I believe it should be a Wine test site maintainers
> responsibility (it's WineHQ web space after all), winrash
> has no direct relationship to Wine development since it's
> not in the Wine CVS.

Yes, it's a problem, I find it pretty hard to understand
some corners of winrash.  But it's mainly that installer
thingie, not the core functionality...

Back to the point, having no way to reliably determine
desktop visibility there's not much I (or even winetest) can
do to effect this.  It's winrash we have to change, I'm
afraid.

invisible desktops - major pain in the testing process

[Ferenc Wagner]

"Dmitry Timoshkov"  writes:
> "Ferenc Wagner"  wrote:
> > I was under the impression that most of the tests are
> > independent of desktop visibility.
>
> Not really. Any API which directly or indirectly creates
> windows or uses GDI is affected by the desktop visibility.

Roughly checking the latest results shows that the following
tests are affected seriously by desktop visibility:

ddraw:ddrawmodes
gdi32:metafile
ntdll:reg
user32:input
user32:msg
user32:sysparams

Some pretty important ones, I agree, but hardly a majority.
> Also, as I already pointed out, Wine doesn't run in that
> mode at all, so we can't compare apples to apples in that
> case.

You are absolutely right in the cases where it makes any
difference (more than the above list)
> And the apps most of the developers/users care about
> should be run on a visible desktop anyway.

Yes.  I don't argue that testing on a visible desktop is a
must.  Some people are actually doing that, it seems.  Or
maybe that's just you alone... :)  Anyway, I tried to make
such results visually distinctive so that we shouldn't drop
all the rest.  Too bad it doesn't work.
> figuring out why it doesn't work is completely useless
> IMO, since the results of the tests running on an
> invisible desktop can't be used for a reasonable
> comparison.

I see your point, even if find it a bit strong.  Ivan's
suggestion is nice, but that requires a probably not so
trivial change in winrash.  I wonder if Chris will do it.
> > And the winrash service is the only way to get several
> > reports quickly for a new test.
>
> Why? What prevents someone to run the tests manually in
> interactive mode once a day?

Nothing.  Somebody is actually doing it for NT4, 2000 and
XP.  I did it now and then for Win98, but this one can't
really be told apart from the invisible ones (not
considering error counts).  From http://winehq.org/site/janitorial:

Volunteers who will run the tests their Windows platform
    of choice on a regular basis so that we quickly fix
    incorrect tests

* Win95: Michael Stefaniuc, Fabian Cenedese
        * Win98: Tony Lambregts, Jeff Smith (SE)
        * WinNT: Luke Stras
        * WinME: James K Whiting
        * Win2k: Tom Wickline, Kye Lewis
        * WinXP: David Miller, Kye Lewis
> If that someone can't or won't do it, then we have to find
> another someone. I'm pretty sure that there are enough not
> lazy people wishing to help we could to choose from.

OK.

invisible desktops - major pain in the testing process

[Chris Morgan]

> Yes.  I don't argue that testing on a visible desktop is a
> must.  Some people are actually doing that, it seems.  Or
> maybe that's just you alone... :)  Anyway, I tried to make
> such results visually distinctive so that we shouldn't drop
> all the rest.  Too bad it doesn't work.
> > figuring out why it doesn't work is completely useless
> > IMO, since the results of the tests running on an
> > invisible desktop can't be used for a reasonable
> > comparison.
>
> I see your point, even if find it a bit strong.  Ivan's
> suggestion is nice, but that requires a probably not so
> trivial change in winrash.  I wonder if Chris will do it.

What change do you want?  The ability for winetest to be able to
interact with the desktop?  I've tried enabling this before but it
didn't seem to have any effect.  I've been quite busy lately and
haven't had the time to keep on on winrash development.  I'd like to
turn over maintenance on winrash to someone else, if anyone is
interested feel free to email me and I can set you up as an admin on
the project.

Chris