Re: crypt32/tests: check Vista error codes for the msg tests.

[Markus Hitter]

Am 23.07.2008 um 02:25 schrieb Reece Dunn:
> - /* last error -- NT: E_INVALIDARG, 9x/Vista: unchanged */
> - /* ret is FALSE on XP and earlier but TRUE on Vista, therefore
>
> it cannot be tested for */
> - ok((GetLastError() == E_INVALIDARG || GetLastError() ==
>
> 0xdeadbeef),
> -
>   "Expected E_INVALIDARG or 0xdeadbeef, 0x%x\n", GetLastError 
>
> ());

A more general question: Is it Wine's policy to just ignore  
differences in behaviour between different Windows versions? From my  
own (naive) standpoint I'd say something like this would be better  
(pseudo-code):

if (WinVer <= XP) {
     ok((!ret && [...]),
        "Expected [...]);
} else {
     ok((ret && [...]),
        "Expected [...]);
}

Likely a often asked question, but I couldn't find hints about the  
answer yet.

MarKus

- - - - - - - - - - - - - - - - - - -
Dipl. Ing. Markus Hitter
http://www.jump-ing.de/

crypt32/tests: check Vista error codes for the msg tests.

[Reece Dunn]

2008/7/23 Markus Hitter :
> Am 23.07.2008 um 02:25 schrieb Reece Dunn:
> > - /* last error -- NT: E_INVALIDARG, 9x/Vista: unchanged */
> > - /* ret is FALSE on XP and earlier but TRUE on Vista, therefore
> >
> > it cannot be tested for */
> > - ok((GetLastError() == E_INVALIDARG || GetLastError() ==
> >
> > 0xdeadbeef),
> > -
> >   "Expected E_INVALIDARG or 0xdeadbeef, 0x%x\n", GetLastError
> >
> > ());
>
> A more general question: Is it Wine's policy to just ignore
> differences in behaviour between different Windows versions? From my
> own (naive) standpoint I'd say something like this would be better
> (pseudo-code):

The tests should not rely on detecting the running platform version.
> if (WinVer <= XP) {
>     ok((!ret && [...]),
>        "Expected [...]);
> } else {
>     ok((ret && [...]),
>        "Expected [...]);
> }

This gets messy when testing something like urlmon, shlwapi and others
that get updated with IE. You then have to check IE version as well,
or more specifically the DLL version.

This would then make it difficult for the tests to pass on Wine that
is essentially a hybrid of all the current Windows versions.
> Likely a often asked question, but I couldn't find hints about the
> answer yet.
> From http://www.winehq.org/site/docs/winedev-guide/testing-platforms:

"If an API is only present on some Windows platforms, then use
LoadLibrary and GetProcAddress to check if it is implemented and
invoke it. Remember, tests must run on all Windows platforms.
Similarly, conformance tests should nor try to correlate the Windows
version returned by GetVersion with whether given APIs are implemented
or not. Again, the goal of Wine is to run Windows applications (which
do not do such checks), and not be a clone of a specific Windows
version. "

- Reece

crypt32/tests: check Vista error codes for the msg tests.

[Juan Lang]

Hi Markus,
> A more general question: Is it Wine's policy to just ignore
> differences in behaviour between different Windows versions?

No, we wish to accept most Windows versions.  Not quite all, because
in some cases old versions of Windows are clearly broken.  That's what
we have the handy broken() for in the tests.
> From my own (naive) standpoint I'd say something like this would be better
> (pseudo-code):
>
> if (WinVer <= XP) {
>     ok((!ret && [...]),
>        "Expected [...]);
> } else {
>     ok((ret && [...]),
>        "Expected [...]);
> }

This isn't better, because the Windows version isn't a reliable way to
check the expected behavior.  New service packs, new versions of IE,
new drivers, and many other things not reflected by the Windows
version affect the expected result.  So the current way of accepting
any Windows result we think is not terribly broken is better than your
suggestion.
> Likely a often asked question, but I couldn't find hints about the
> answer yet.

Indeed, that is often asked.  /me thinks we need a developer FAQ..
--Juan