-----Original Message----- From: Alexandre Julliard julliard@winehq.com {WineHQ-devel}
Version checks are a bad idea because this is not how Wine is implemented. For instance GetLongPathName under Wine doesn't try to fail if you are running with -winver win95, it just works. Adding version checks in the tests just means that some functions won't get tested if you run with the wrong version.
The right way is to always call the function no matter the version, and check that the call either succeeds as expected, or fails in the way that it's supposed to fail on a platform that doesn't support it.
The problem I forsee with this is that it makes writing tests a bit tricky for functions that behave differently on different platforms (i.e. defining the failure conditions correctly). Otherwise, it is a good thing to strive for.
I'll try to modify the tests I've written to behave in this manner, and see how much work it is. Unfortunately, it is somewhat difficult, since I don't (in general) have access to the platforms which don't support the functions. So I can write tests that pass on Win2k, Win98, and wine, but as soon as someone runs it on win95 or nt4.0, it will likely fail (since I have no idea what the failure mechanism will be on these platforms, and thus can't gaurd against it) until someone can provide enough feedback to allow the test to be fixed.
My assumption has always been: Make the function work correctly on Windows, and then add todo's to support wine. Thus I've put the effort in upfront to catch errors when the application runs on an MS OS. It sounds to me that you are proposing a more wine-centric testing base (make the tests work in Wine, and fix problems encountered in Windows as they pop up...or be a lot more knowledgeable than me at how Windows is supposed to behave). So far I have written all my tests based purely on the MSDN spec, and then tweaked them when an OS behaves differently than documented. If I don't use OS-specific checks, I just feel like I'm leaving a known whole in my test if I don't account for the the cases which are documented not to work (and which I don't know how they will behave).
For instance the thread.c test that I wrote makes (some) checks for access control, which only work on win2k+. The way I wrote the test, it just worked in win98 when I finally got around to testing it. Had I just executed the checks, I would have had no idea how Win98 would behave (hopefully with a nice SetError, but god only knows), and the test would certainly have failed on win98 until I was able to try it and figure out how it dealt with this unsupported functionality.
Lastly, what is the recomendation for dealing with cases like this: lstrcpyA(tmpstr,"aaaaaaaa"); len1=GetTempPathA(len,tmpstr); /*this should fail because len is too small */ ok(len1==len+1, "GetTempPathA should return string length %d instead of %d",len+1,len1); if(WIN2K_PLUS(version)) { /* in Win2k, the path won't be modified, but in win98, wine it is */ todo_wine { ok(lstrcmpiA(tmpstr,"aaaaaaaa")==0, "GetTempPathA should not have modified the buffer"); } } One answer is not to test this, since it is not documented what the result will be, but the OSes certainly behave differently, and it is possible that some program takes advantage of this. I'm sure there are other similar cases, and I'd like to know (in general) what the right solution will be.
I'm probably making a mountain out of a molehill, and I'll just try modifying my tests to work the way Alexandre proposes. I'm just afraid this is setting additonal obstacles for (non Wine) developers to write regression tests.
Thanks, .Geoff