[Bug 21773] New: Mail address needed to run dotests but no means to provide it
http://bugs.winehq.org/show_bug.cgi?id=21773 Summary: Mail address needed to run dotests but no means to provide it Product: Wine Version: unspecified Platform: x86 OS/Version: Linux Status: UNCONFIRMED Severity: normal Priority: P2 Component: testcases AssignedTo: wine-bugs(a)winehq.org ReportedBy: iang(a)austonley.org.uk The test suite now requires an email address. In itself this is a good idea. However, there is no provision in the dotests script for the user to provide an email address, only a tag. In consequence the script fails to run. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 Austin English <austinenglish(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|UNCONFIRMED |RESOLVED Resolution| |INVALID --- Comment #1 from Austin English <austinenglish(a)gmail.com> 2010-02-20 10:20:25 --- That script is deprecated. Use winetest-latest.exe directly. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 Austin English <austinenglish(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |CLOSED --- Comment #2 from Austin English <austinenglish(a)gmail.com> 2010-02-20 10:20:47 --- Closing. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 --- Comment #3 from Ian Goddard <iang(a)austonley.org.uk> 2010-02-21 07:56:59 --- I always build wine from the source tree as I have to patch it to work round a feature which for me is a show-stopping bug. If I run dotests (and, BTW, how is anyone supposed to know it has been deprecated if it's still sitting there in the source tree?) it runs against the new build before that build has been make installed. *Only* if it looks like the build won't cause trouble (and in the past several hung in dotests so severely that I had to power-cycle the laptop) will I patch it and then make install it. AFAICS winetest-latest.exe will be run using the currently make installed version of wine - which, ATM, is the *previous* version. I'm not going to make install an untested build. But I can't now do that using dotests. So I now have to do it in two stages, first run make test -k and, if it looks OK I can make install, run winetest-latest and then patch, rerun make and make install. However make test -k has just hung in one of the tests. I've no idea if, were it running in dotests or in winetest-latest, it would hang so severely that I wouldn't install the build. But, knowing that there's a problem but not being able to find out how severe, there's no way I'm going to run make install so in turn I won't be running winetest-latest against the build so you'll never get a test report on it and hence there's no way that you'll be able to get a handle on the problem which the test exposed. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 --- Comment #4 from Dmitry Timoshkov <dmitry(a)codeweavers.com> 2010-02-23 10:49:08 --- (In reply to comment #3)
I always build wine from the source tree as I have to patch it to work round a feature which for me is a show-stopping bug.
You can't post the test results for Wine with a custom patch, any custom patch makes the test results invalid. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 --- Comment #5 from Ian Goddard <iang(a)austonley.org.uk> 2010-02-23 11:09:23 --- (In reply to comment #4)
(In reply to comment #3)
I always build wine from the source tree as I have to patch it to work round a feature which for me is a show-stopping bug.
You can't post the test results for Wine with a custom patch, any custom patch makes the test results invalid.
Read this sentence again: "So I now have to do it in two stages, first run make test -k and, if it looks OK I can make install, run winetest-latest and then patch, rerun make and make install." Note the sequence in which the steps "run winetest-latest" and "patch" occur. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 --- Comment #6 from Alexandre Julliard <julliard(a)winehq.org> 2010-02-23 11:15:34 --- It's not clear what you are trying to do, but simply doing "./wine winetest-latest.exe" inside the build tree runs winetest on that tree. You can also do "./wine winetest" which run the Winelib version of winetest. Either way you don't need dotests. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 --- Comment #7 from Austin English <austinenglish(a)gmail.com> 2010-02-23 12:11:03 --- (In reply to comment #3)
If I run dotests (and, BTW, how is anyone supposed to know it has been deprecated if it's still sitting there in the source tree?)
It's not in the source tree. You downloaded from the wiki and put it in your source tree. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 --- Comment #8 from Ian Goddard <iang(a)austonley.org.uk> 2010-02-26 11:54:05 --- (In reply to comment #6)
It's not clear what you are trying to do
I'm trying to do two things at the same time. One is to provide you with regression testing and the other is to provide myself with user acceptance testing. I hope we can both agree that best practice in testing starts with setting up a testing environment (sandbox) separate from the production environment so that neither can interfere with the other. The instructions on the website for running wintest-latest will simply pick up the current wine version in the user's $PATH so it seems to be implied that the user has already installed the latest wine into the production environment. In other words, install first, test later. I hope I don't have to labour the point of how far this fails to meet best practice. If you have to consider deprecating something, this is a prime candidate. On first glance your suggestion of running ./wine in the source tree seems OK in that it doesn't involve installing before test. However there's nothing to guarantee either of us that test and live environments won't interfere.
From your point of view you don't know that I don't have $WINEwhatever environment variables set. If I do then the tests my pick up something from my production environment, for instance a registry setting or a dll which may affect testing.
From my PoV I don't know, even if I don't have such environment variables set, that your S/W won't make some default assumptions, pick up my production environment and disturb it in some way.
Clearly better practice would be to use a script which would set its own environment variables. Just as dotests does. Maybe you should consider not only reprecating it, to coin a word, but making it the norm. Ian -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 --- Comment #9 from Ian Goddard <iang(a)austonley.org.uk> 2010-02-26 11:57:52 --- (In reply to comment #7)
(In reply to comment #3)
If I run dotests (and, BTW, how is anyone supposed to know it has been deprecated if it's still sitting there in the source tree?)
It's not in the source tree. You downloaded from the wiki and put it in your source tree.
You're right! It's so long since I did this that I'd forgotten. However, if it really is to be deprecated (and read my other post on this) then surely the best thing to do would be to put it in the source tree to overwrite the original version with one to announce that it's deprecated and to give clear instructions on what to do instead. Otherwise, my question still stands in essence: how are users to know until, as in this instance, it fails? Ian -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 --- Comment #10 from Vitaliy Margolen <vitaliy(a)kievinfo.com> 2010-02-26 23:51:07 --- You've been given number of ways how you can run tests within newly compiled environment: 1. ./wine /path/to/winetest-latest.exe 2. ./wine winetest 3. make test All of the above _guarantees_ to use wine from the current source tree and not system installed one. Of course in all cases you have to start with clean wineprefix and make sure wine_gecko is available. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 --- Comment #11 from Ian Goddard <iang(a)austonley.org.uk> 2010-02-27 06:06:19 --- (In reply to comment #10)
You've been given number of ways how you can run tests within newly compiled environment: 1. ./wine /path/to/winetest-latest.exe 2. ./wine winetest 3. make test
All of the above _guarantees_ to use wine from the current source tree and not system installed one. Of course in all cases you have to start with clean wineprefix and make sure wine_gecko is available.
Just read your last sentence again: "Of course in all cases you have to start with clean wineprefix and make sure wine_gecko is available." If you simply tell people to take any of the above 3 you do not assure that the proviso in that sentence will be met. That means that you cannot trust any of the regression-testing results you get back from the community. Simply using the wine from the current source tree isn't enough. Good testing practice just isn't that simple. Ian -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 --- Comment #12 from Alexandre Julliard <julliard(a)winehq.org> 2010-02-27 07:22:45 --- (In reply to comment #11)
(In reply to comment #10) Just read your last sentence again: "Of course in all cases you have to start with clean wineprefix and make sure wine_gecko is available."
If you simply tell people to take any of the above 3 you do not assure that the proviso in that sentence will be met. That means that you cannot trust any of the regression-testing results you get back from the community.
Simply using the wine from the current source tree isn't enough. Good testing practice just isn't that simple.
The goal of the tests is not to run against an ideal perfect setup that is always identical. Getting a variety of tests run against a variety of setups, some less correct than others, is much more useful for spotting problems. So if you want your own systematic always-reproducible test environment, feel free to write your own script to ensure that. But distributing the script for everybody to use would actually make the test results less useful. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 --- Comment #13 from Ian Goddard <iang(a)austonley.org.uk> 2010-02-27 07:54:42 --- (In reply to comment #12)
(In reply to comment #11)
(In reply to comment #10) Just read your last sentence again: "Of course in all cases you have to start with clean wineprefix and make sure wine_gecko is available."
If you simply tell people to take any of the above 3 you do not assure that the proviso in that sentence will be met. That means that you cannot trust any of the regression-testing results you get back from the community.
Simply using the wine from the current source tree isn't enough. Good testing practice just isn't that simple.
The goal of the tests is not to run against an ideal perfect setup that is always identical. Getting a variety of tests run against a variety of setups, some less correct than others, is much more useful for spotting problems.
I understand that, at least as regards a variety of H/W and O/S platforms and the test system reports such platform details back to you. However, as I understand it, if $WINEPREFIX were not clean but pointed to the current, older, production installation the test could wander off and pick up dlls from that installation. If that happens the results reported back to you would largely reflect that version. In a production installation such as mine you'd be seeing the results of a patched winex11.drv. It may be that you're seeing more variety than you think! Ian -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=21773 --- Comment #14 from Alexandre Julliard <julliard(a)winehq.org> 2010-02-27 13:25:13 --- The version of the dlls being used doesn't depend on the wine prefix. And if you are configured to use native dlls, winetest detects and reports it. Strange as it may seem, we have already thought about these issues. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
participants (1)
-
wine-bugs＠winehq.org