Ben Klein wrote:
2009/4/13 chris ahrendt celticht32@aol.com:
Vincent Povirk wrote:
On Sun, Apr 12, 2009 at 5:24 PM, Ben Klein shacklein@gmail.com wrote:
2009/4/13 Vincent Povirk madewokherd+8cd9@gmail.com:
But the description doesn't say "invalid conditions". It says "serious errors in Wine". That's something that should never happen in tests, as it would imply that the state of the libraries we're testing (and thus the test) is invalid.
How about "serious errors in Wine, or in the test cases, sometimes deliberate errors in the tests"? As Vitaliy points out, some tests deliberately do invalid things to make sure they fail. We can't ONLY test on things that succeed!
I'm not against changing the description. If it's OK to have err messages for cases that we know fail (but don't crash or prevent the library from continuing to function), then the Developer's Guide is wrong and should be changed. I also don't care how long it takes to make the change, as long as there's a consensus that it's the guide that's wrong and not the reality of what's in Wine.
This is EXACTLY the point I am trying to get to.. if they are normal and not ERRORS but warnings then they should be thus and noted in the developers guide. Right now Wine isn't even following its own guidelines in this case.
No. Not warnings. Errors. They are errors. There is no way to distinguish an error caused by a real application from an error caused by a Wine test.
If the situation is an error and it is expected, the test should handle this, like:
ok (some_test_here_that_should_fail, "The test that should fail, did/n")
I'm guessing that most of the tests that should fail, do. I don't know if there is a failure like there is an ok.
If you don't like it, run all your tests with WINEDEBUG=-all.
And that will prove nothing. Tests should be run with debugging on. You are really being sarcastic, right?
As to the discussion, I will add my .02 Eurocent here:
Fixme: Code that is not implemented that should be. Warning: Code that encountered an unexpected or new condition that is not normal. Data should not be corrupted by this event. Error: Code encountered a condition that will result in corruption of data. It appears that we have 'error creep' and that is not good from a user point of view and it is really irritating from a support point of view.
During testing an error could be either a Warning or an Error. Tests should not be run against non-existent Wine code, but should against existing Windows code. The situation with testing is that unexpected or improper behavior of code should be an error. There is no such thing as a warning during a test run. Either the test passes, which means that Wine is acting the same as a certain version of Windows, or it is not.
Now, the problem is that we are sending cryptic messages to end users, most of which they can and should ignore. Err messages piling up on their terminal windows should be a cause for concern. If we know that a condition does not cause data corruption, then we should not be marking it as an error, but maybe a warning or if the code is non-existent or improper, a fixme.
Can we start to clean up some of the most obvious "it is not an error but the message says it is" so that we can calm down users who encounter them?
James McKenzie