Scott Ritchie wrote:
There was a bit of a philosophical discussion on #winehackers about the merits of creating tests for functions that might be testing undefined or unimportant behavior. Windows behaves one way, we behave another, the tests measure this delta, but it's unknown if this will actually improve a real world application.
Vincent Povirk wrote an excellent reply. I hope somebody will turn that into a Wiki page.
More broadly, should we resist any change without a particular (real-world) application in hand that needs it?
This is stupid because you're always late. Like a fireman. a) Somebody who wrote the bug report is waiting for the bug to be fixed ASAP. b) It takes a lot of effort to analyse huge and cryptic log files to find out what the particular delta is.
Or should we err on the side of testable behavior,
What you test is what you know. It's not erring. If you don't know what the behaviour should be, you can't add a FIXME pointing out that Wine deviates from native behaviour.
When I asked about testing depth w.r.t. the MCI a few years ago, Paul Vriens advised to test to the bones. IMHO that's good advice.
I recommend beginners to start writing tests *first*. Doing so, you learn a lot about the component Wine purports to mimic. You can then even start to predict failures!
Somehow I believe I'm a very untypical Wine contributor. I'm not bug driven. Quite to the contrary, my thesis is that once you've tested a component to the bones *and* replicated the observable behaviour, there will be few opportunities left for people to report bugs.
The basic alternative is that if you have free time now, you can either try to understand obscure logs of some component you're not exactly familiar with and more or less well documented, perhaps single-step your way through the debugger, *OR* you can spend time shedding light on said component via tests, mostly working with logs you control because you're working with full source to your test and the Wine code.
That's what I was doing with the audio parts of the MCI before WinMM was rewritten to use mmdevapi. Now that's what I'm doing with mmdevapi. After that, should I still have the energy, I'll look at WinMM audio.
Other benefit: - Tests get executed daily whereas an application X can be made to work in wine-X yet break again in X+1 until somebody notices.
Regards, Jörg Höhle