Cascading test failures
Ideally, tests should be independent on each other. That's not the case right now. A clear example is msi:package, which often fails (on XP) with 1574 errors, see http://test.winehq.org/data/tests/msi:package.html. For all the cases I've checked, when these 1574 errors occur the preceding msi:install test timed out (the reverse is not true, I see msi:install timeouts followed by a successful msi:package). My current theory is that the Windows Installer service gets confused when the msi:install test times out and is bluntly terminated by the winetest wrapper. A possible solution might be to introduce a cleanup mechanism invoked by winetest.exe when it detects a timeout or crash in one of the test executables. After terminating the test executable, winetest.exe would then call "testname_test.exe --cleanup failed_test_name". The --cleanup argument could be handled by the tests main(), just like --list is handled by main(). A test executable would be able to register a cleanup function by setting a global function pointer to its cleanup function. In the case of msi_test.exe, that cleanup function could try to stop and restart the Windows Installer service. Does this sound worthwhile to pursue? Any other/better ideas? Ge.
On Fri, 2010-07-16 at 04:36 -0700, Greg Geldorp wrote:
For all the cases I've checked, when these 1574 errors occur the preceding msi:install test timed out (the reverse is not true, I see msi:install timeouts followed by a successful msi:package).
Do you have any clue why the test times out? Is there a dialog waiting to be clicked?
On 07/16/2010 03:05 PM, Hans Leidekker wrote:
On Fri, 2010-07-16 at 04:36 -0700, Greg Geldorp wrote:
For all the cases I've checked, when these 1574 errors occur the preceding msi:install test timed out (the reverse is not true, I see msi:install timeouts followed by a successful msi:package).
Do you have any clue why the test times out? Is there a dialog waiting to be clicked?
In the past we have seen that on slower machines the number of tests just exceeded the timeout. That has been 'corrected' back than by making some tests interactive. I guess because of the multitude of new tests we have ended up in the same situation. -- Cheers, Paul.
On Fri, 2010-07-16 at 15:12 +0200, Paul Vriens wrote:
In the past we have seen that on slower machines the number of tests just exceeded the timeout. That has been 'corrected' back than by making some tests interactive. I guess because of the multitude of new tests we have ended up in the same situation.
The msi:install test completes in 90 seconds here in a vm, so well within the timeout of 300 seconds.
From: Hans Leidekker [mailto:hans(a)meelstraat.net]
On Fri, 2010-07-16 at 15:12 +0200, Paul Vriens wrote:
In the past we have seen that on slower machines the number of tests just exceeded the timeout. That has been 'corrected' back than by making some tests interactive. I guess because of the multitude of new tests we have ended up in the same situation.
The msi:install test completes in 90 seconds here in a vm, so well within the timeout of 300 seconds.
The winetest.exe timeout is 120 seconds, not 300 (TestBot uses 300). So 90 seconds isn't that far off from the timeout. The MsiInstallProductA() call in test_delete_services() consistently took 30 seconds (25% of the available time) during my testing. Ge.
On Fri, Jul 16, 2010 at 11:35 PM, Hans Leidekker <hans(a)meelstraat.net> wrote:
On Fri, 2010-07-16 at 15:12 +0200, Paul Vriens wrote:
In the past we have seen that on slower machines the number of tests just exceeded the timeout. That has been 'corrected' back than by making some tests interactive. I guess because of the multitude of new tests we have ended up in the same situation.
The msi:install test completes in 90 seconds here in a vm, so well within the timeout of 300 seconds.
I haven't run a full winetest for a few weeks, but I just gave msi:install a run on my laptop, Vista 32bit SP2, Intel Core 2 Duo @ 2.53GHz, 4GB ram. install: 8535 tests executed (0 marked as todo, 76 failures), 1 skipped. Took 4 minutes 47 seconds, as another data point.
Hans Leidekker <hans(a)meelstraat.net> writes:
On Fri, 2010-07-16 at 04:36 -0700, Greg Geldorp wrote:
For all the cases I've checked, when these 1574 errors occur the preceding msi:install test timed out (the reverse is not true, I see msi:install timeouts followed by a successful msi:package).
Do you have any clue why the test times out? Is there a dialog waiting to be clicked?
Yes, it seems the install test often triggers dialogs. -- Alexandre Julliard julliard(a)winehq.org
participants (5)
-
Alexandre Julliard -
Greg Geldorp -
Hans Leidekker -
Jeff Zaroyko -
Paul Vriens