James wrote:
Looking at the test data, all of the msi:install tests timeout. I just ran the install tests in XP running under vmware on a 3ghz machine. The tests took 9m41s. That completely blows away the 2min timeout. There's nothing wrong with the tests, they just take a long time. I don't think we should extend the timeout, because it's very subjective and more tests will be added, meaning we'll have to change the timeout eventually. I do think we should have a flag or variable that allows the timeout to be ignored for certain tests. Any opinions?
I'd like a way to specify the expected runtime, for use on tests that we observe take a long time. It would be used in two ways: 1) runtest could take an option --skip-long-tests which would skip all tests that had that option set, and 2) by default runtest would only abort those tests if they took one minute over their expected time (say).
That would give us both a quick interactive make test and a more reliable slow-but-complete make test. - Dan