On 08/26/2011 12:45 PM, Joerg-Cyril.Hoehle@t-systems.com wrote:
Scott Ritchie wrote:
There was a bit of a philosophical discussion on #winehackers about the merits of creating tests for functions that might be testing undefined or unimportant behavior. Windows behaves one way, we behave another, the tests measure this delta, but it's unknown if this will actually improve a real world application.
Vincent Povirk wrote an excellent reply. I hope somebody will turn that into a Wiki page.
More broadly, should we resist any change without a particular (real-world) application in hand that needs it?
This is stupid because you're always late. Like a fireman. a) Somebody who wrote the bug report is waiting for the bug to be fixed ASAP. b) It takes a lot of effort to analyse huge and cryptic log files to find out what the particular delta is.
Or should we err on the side of testable behavior,
What you test is what you know. It's not erring. If you don't know what the behaviour should be, you can't add a FIXME pointing out that Wine deviates from native behaviour.
When I asked about testing depth w.r.t. the MCI a few years ago, Paul Vriens advised to test to the bones. IMHO that's good advice.
It is a good advice. But that doesn't means one has to submit tests that don't make sense.
I recommend beginners to start writing tests *first*. Doing so, you learn a lot about the component Wine purports to mimic. You can then even start to predict failures!
Somehow I believe I'm a very untypical Wine contributor. I'm not bug
I'm not bug driven either,
driven. Quite to the contrary, my thesis is that once you've tested a component to the bones *and* replicated the observable behaviour,
but here I heartily disagree. You don't test nor replicate all the observable behavior; that is way way too broad. An extreme case would be to try to test how much time/CPU cycles a function call takes and trying to replicate that in Wine. In general that doesn't matter but is is observable and testable.
What you do want to test is: - the de jure API (as documented), and - the de facto API (as used by applications and thus kept "broken" across Windows version).
So if an application relies on the fact that a function call will crash with an invalid combination of arguments than Wine will have to crash too but it doesn't matter if the segfault happens at address 0x00000002 or 0x00000001.
there will be few opportunities left for people to report bugs.
You sub-estimate the ingenuity of application developers to misuse the API. Or you found a way to solve the halting problem ;)
The basic alternative is that if you have free time now, you can either try to understand obscure logs of some component you're not exactly familiar with and more or less well documented, perhaps single-step your way through the debugger, *OR* you can spend time shedding light on said component via tests, mostly working with logs you control because you're working with full source to your test and the Wine code.
It is not an either or but a "do both", for the project in general that is. People are free to scratch their itch as long as they don't make the code a mess.
bye michael