2008/10/31 Dan Kegel dank@kegel.com:
The public internet can (and does) go pear-shaped in the middle of test runs, which means any test that tries to access the public internet is de facto flaky, even if it tries to protect itself by skipping if the internet is down.
So let's provide a way for automated testers to force all hostname resolution requests to return the same result (say, 'localhost' or 'mytestserver'). Only include this in debugging builds.
Later we should provide a script to set up a local test server that can handle all of our conformance tests' requests, but for now, just letting patchwatcher or winetest redirect everything to localhost will let tests fail reliably instead of randomly. See http://bugs.winehq.org/show_bug.cgi?id=15716
This approach won't help users of the Wine conformance tests running on Windows, so it appears to me to be a flawed approach. We should instead rewrite any existing tests that use a public server to instead use a server started by the Wine test code (see dlls/wininet/tests/http.c:server_thread). This has the additional benefit of allowing us to be much more precise with what we're testing and to be able to emulate the implementations of different HTTP servers, instead of just what winehq.org runs.
-
Rob Shearman