On Sun, May 18, 2008 at 10:02 PM, Dmitry Timoshkov dmitry@codeweavers.com wrote:
How much tests do you usually build, and how often? I do it routinely almost every day on a separate Windows machine by copying single .c files and compiling them as standalone tests. I don't see any rationale behind your suggestion.
Every time I do a build I try make test. Thats often enough and every time it fails and has for years. This should have been unacceptable 5 years ago and should unacceptable now. I don't care what causes the failure, if its a broken third party application like metacity or a soft dependency missing. This complacency has kept the framework from passing 100% for anyone on anything other than Alexandre's box. At the very least we should have targeted a certain version of Windows years ago and said "It is unacceptable for the framework to ever fail here" but I digress. To return to your question, as I am rejoining the ReactOS project on June 1st I plan on doing a lot more work with winetest as per Alexandre's statement years ago that ReactOS contributions to the test suite would be allowed.
Let me try to make it more clear. If the person compiling Wine does not have a certain library say libxml2, Wine will still build a stub msxml3.dll and build tests but those tests are going to fail when that person does make test. If msxml3 is broken by the lack of libxml2 then the tests need to reflect this by skipping.
Not everyone supposed to run the Wine tests, only those do who *really* can comprehend the results and able to do something with them.
NO. Everyone that builds Wine from source should run make test. It should pass 100% for everyone everywhere. If does not then there is something wrong. It does the world not 1 damn bit of good if passes on your system and no one elses.
Again, the tests do not target *anyone*, target audience of Wine tests is Wine developers who is able to cope with the results. Anyone else should play somewhere else.
Again, if they are building wine from source and smart enough to run make, then they are smart enough to run make test or winetest and if something fails, they or we need to know about it and know why. We should not accept anything less than a 100% pass rate everywhere.
If you are deveoping a winelib application I don't think you want to run the tests at all. I don't really matter whether the tests pass for you.
This is asinine. Everyone that has ever developed a Winelib application has at some point or another needed to fix some area of Wine. If they are working on Wine code then they are going to need to run the framework.
It doesn't matter how hard we try to stop broken builds, there is not way to stop creating them by an incompetent person. Same true for say kernel builds, and any other project.
If the build is broken by the result of a certain library then it should be a hard dependency and the Wine build should fail. Not the bloody testing framework.