On 10/25/07, James Hawkins truiken@gmail.com wrote:
- runtest could take an option --skip-long-tests
which would skip all tests that had that option set, and
I don't think that's fair to long tests, say msi:install. There will always be people that don't want to wait for the tests, and thus the long tests get less exposure.
But it's common practice in the testing community to offer a "quick smoke test" along with an exhaustive test. I'm just proposing we do that. If "make test" is fast enough, people will use it more frequently. I think that makes up for the "long tests aren't run as often" effect.
In the case of msi:install, there's technically nothing wrong with the tests, so it should get just as much exposure as any other test. Just today I noticed a bug in msi:install that I didn't know was there before because the tests were timing out. Like I said before, we shouldn't specify a limit for certain tests like msi:install because they'll always get longer and we'll always be chasing a moving target.