Please both calm down a bit.
dmitry@codeweavers.com wrote:
"Steven Edwards" winehacker@gmail.com wrote:
dmitry@codeweavers.com wrote:
Not everyone supposed to run the Wine tests, only those do who *really* can comprehend the results and able to do something with them.
NO. Everyone that builds Wine from source should run make test.
Why? Does it magically make something useful? Again, what about other packages built from source?
To limit running of the test frame work only to those that know exactly what they are doing is a to strict limitation IMHO.
You should not forget that acceptance of a patch is based on the criteria that it runs make test. Which in may case has been always a limited make test in the directory itself only, since a whole make test seldom or never even got so far as to run for the component I was working at.
While sending patches is of course the right thing to do, it has been and still is rather discouraging to get wine sources, building them on ones own box and finally run a make test and find that there are many areas where it simply reports failures or crashes altogether especially in areas I do not have the knowledge to do anything about and not the time to dig into much deeper.
The fact that I couldn't get the whole make test to run out of the box on my system without bad crashes and/or numerous test failures has not only made me not run it anymore except in tree for the component I attempted to create a fix for, but has actually discouraged me in the past from working on more bug fixes or improvements other than what I have found strictly necessary to make a specific app working.
It should pass 100% for everyone everywhere. If does not then there is something wrong. It does the world not 1 damn bit of good if passes on your system and no one elses.
If the test doesn't pass for you, go ahead and fix it. I can't do that for you if the test passes for me.
But isn't that the crux of why this discussion started? Someone finding that a soft dependency of Wine on a library should not create a failure but a skip when that dependency was not satisifed and this was detectable in a sane way. And getting shot down that this system should be considerd broken.
If the missing soft dependency should be considered a broken system then the dependency should be not soft and vice versa.
I personally don't mind a test failure in a library that depends on something I know I don't use and consequently don't have installed but if it's possible to skip the tests for that why not do it?
The test framework should not only help avoiding regressions for the VERY active developers but allow anyone working on Wine in one way or the other to have some confidence that he got it to build right. It's what I usually do when installing any source package to see if I missed something. Getting a long list of errors and sometimes crashes out of the box really leaves a bad taste and also makes it much harder to verify that my own modifications to it later on didn't break it in some ways.
Rolf Kalbermatter
"Rolf Kalbermatter" r.kalbermatter@hccnet.nl wrote:
The test framework should not only help avoiding regressions for the VERY active developers but allow anyone working on Wine in one way or the other to have some confidence that he got it to build right. It's what I usually do when installing any source package to see if I missed something. Getting a long list of errors and sometimes crashes out of the box really leaves a bad taste and also makes it much harder to verify that my own modifications to it later on didn't break it in some ways.
That's the whole reason behind http://wiki.winehq.org/MakeTestFailures
Carefully investigating each test failure one by one as Dan does is the right approach IMO, flaming on wine-devel is not.
On Mon, May 19, 2008 at 1:47 AM, Dmitry Timoshkov dmitry@codeweavers.com wrote:
Carefully investigating each test failure one by one as Dan does is the right approach IMO, flaming on wine-devel is not.
Sorry if I went a little too flamish in my reply earlier. I did not mean to imply your point of view was stupid or used the word asinine it was not addressed at you directly but the general concept that we have allowed it to be broken for so long.
I offered a proposed solution to a whole class of failures and you shot it down saying it was the wrong solution for standalone builds. I addressed that, your answer is now "If some test fails that doen't mean that the build is broken". But I don't understand the logic. The metric should be, if make test fails, the build cannot be assumed safe. Maybe its not really broken but without a 100% pass rate there is no way to assume anything other than brokenness. Even standalone is not a safe test given the current framework as it has 10% failure rate on Windows Server 2003. Maybe it passes for you on XP, maybe its broken for me on Vista. Nothing is safe to assume while allow whole classes of failures. If they really are failures and certain missing librares makes a build "broken" then it should not be a soft dependency.
"Steven Edwards" winehacker@gmail.com wrote:
I offered a proposed solution to a whole class of failures and you shot it down saying it was the wrong solution for standalone builds. I addressed that, your answer is now "If some test fails that doen't mean that the build is broken". But I don't understand the logic. The metric should be, if make test fails, the build cannot be assumed safe.
The logic is that it's perfectly valid to build Wine without OpenGL or XML libraries if the user is not intending to run applications depending on them. If you need them, configure helpfully prints a warning. If you ignore the warning, that's your problem.
Maybe its not really broken but without a 100% pass rate there is no way to assume anything other than brokenness. Even standalone is not a safe test given the current framework as it has 10% failure rate on Windows Server 2003. Maybe it passes for you on XP, maybe its broken for me on Vista. Nothing is safe to assume while allow whole classes of failures. If they really are failures and certain missing librares makes a build "broken" then it should not be a soft dependency.
If you would closely followed wine-patches mailing list, you would notice that *a lot* of efforts lately has been spent on making various Wine tests pass cleanly on different Windows flavours. That very tedious and time consuming work is being done without any fuss, flaming or self advertisement.
-
Dmitry Timoshkov -
Rolf Kalbermatter -
Steven Edwards