[Bug 15716] New: Wine testcases should not use the public internet, as it causes flakiness
http://bugs.winehq.org/show_bug.cgi?id=15716 Summary: Wine testcases should not use the public internet, as it causes flakiness Product: Wine Version: 1.1.6 Platform: Other OS/Version: other Status: NEW Severity: normal Priority: P2 Component: testcases AssignedTo: wine-bugs(a)winehq.org ReportedBy: dank(a)kegel.com Just about every day, patchwatcher rejects a patch incorrectly because some conformance test is relying on the Internet to be there. Sadly, the Internet is not 100% reliable, and therefore must not be used by any test suite that needs to be 100% reliable. We must expunge all use of the internet from our test suite, in favor of mock servers running locally. One way to start would be to add a configure-time option to cause all hostname resolution to return 127.0.0.1. Then people who needed ultrarepeatable conformance test results (like patchwatcher) could use that option and never have flaky results due to the public internet. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=15716 --- Comment #1 from Dan Kegel <dank(a)kegel.com> 2008-10-28 09:13:58 --- Also, the current practice of checking early in the test to see if the net is available, and skipping the test if it's not, is not sufficient, because the net could go down after this check. We need our tests to be thoroughly insulated from the public internet; a bandaid like that is fine for development, but insufficient for production testing. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=15716 --- Comment #2 from Austin English <austinenglish(a)gmail.com> 2008-10-28 12:05:18 --- (In reply to comment #1)
Also, the current practice of checking early in the test to see if the net is available, and skipping the test if it's not, is not sufficient, because the net could go down after this check. We need our tests to be thoroughly insulated from the public internet; a bandaid like that is fine for development, but insufficient for production testing.
Why not install Apache on the patchwatcher box and edit /etc/hosts to point to that local server? -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=15716 --- Comment #3 from Dan Kegel <dank(a)kegel.com> 2008-10-28 12:12:07 --- This isn't really patchwatcher specific. What we need is an easy script that anybody can run that starts all the needed services. As new tests are added that need new services, the developers who add those tests should add the setup for the needed services to the server setup script. Alternately, we could have each test set up its own mock server. (That's harder, but is fairly common practice in many projects.) -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=15716 --- Comment #4 from Dan Kegel <dank(a)kegel.com> 2008-10-28 12:13:51 --- Ideally, the script should run well on either cygwin or ubuntu. That would let people who run the winetest executable on windows give good results even if their net connection is flaky, and raise the quality of the results reported to test.winehq.org. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=15716 Austin English <austinenglish(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |austinenglish(a)gmail.com Keywords| |source, testcase -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=15716 Jeff Zaroyko <jeffz(a)jeffz.name> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jeffz(a)jeffz.name --- Comment #5 from Jeff Zaroyko <jeffz(a)jeffz.name> 2008-10-28 20:35:36 --- I might mention here that I submit results to test.winehq.org while running winetest without an internet connection on a win98 machine and now also w2k8. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=15716 --- Comment #6 from Dan Kegel <dank(a)kegel.com> 2008-10-28 21:54:56 --- Flaky connections can produce lots more failures than absent ones. This bug is about protecting us from flaky connections, so it's neither here nor there how flaky tests are when there is no internet connection. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=15716 Saulius K. <saulius2(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |saulius2(a)gmail.com -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=15716 butraxz(a)gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |butraxz(a)gmail.com --- Comment #7 from butraxz(a)gmail.com 2012-05-18 05:59:25 CDT --- This bug has not been updated for four years. Is this still an issue i current (1.5.4) or newer wine ? You may also close this as abandoned if you feel that that this is issue is no longer relevant to you. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=15716 roger(a)mailinator.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |roger(a)mailinator.com --- Comment #8 from roger(a)mailinator.com --- Last response Dan Kegel 2008-10-28 21:54:56 CDT what do you want to do about this issue? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=15716 François Gouget <fgouget(a)codeweavers.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fgouget(a)codeweavers.com --- Comment #9 from François Gouget <fgouget(a)codeweavers.com> --- Jacek can probably provide more details but my understanding is that we have SSL tests that require connecting to a server having a valid (and possibly various kinds of invalid) SSL certificates. That's all pretty hard to do with a local server. So I don't think this is possible. I'm also not convinced that the reliability of the Internet connection is a significant issue. It seems to be fine for the WineTestBot for instance (at least from the tests side). -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
participants (1)
-
wine-bugs＠winehq.org