Re: C testing framewok. Updated. SystemParametersInfo unit test.
Alexandre Julliard wrote:
You don't need to set the variable when running make test, but you have the possibility to set it if you want to override the default for whatever reason. I don't see why we should impose a less flexible solution that won't be more robust anyway.
Alexandre, by your suggestion I implemented the switch, but left automated platform detection in place. Platform detection will work when the environment variable is not set or set to unknown value. Can you suggest more reliable way to detect if test runs on Wine? One more important change in C framework is support of Unicode in testing. Dimitry, can you look at it? See how it is used in sysparams.c, search for "has_unicode" function call. I'm interested in your comments. Andriy Palamarchuk __________________________________________________ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/
"Andriy Palamarchuk" <apa3a(a)yahoo.com> wrote:
One more important change in C framework is support of Unicode in testing. Dimitry, can you look at it? See how it is used in sysparams.c, search for "has_unicode" function call. I'm interested in your comments.
I would suggest to explicitly use A and W suffixes to avoid confusion: not just SystemParametersInfo, but SystemParametersInfoA. -- Dmitry.
On Thu, 17 Jan 2002, Dmitry Timoshkov wrote:
I would suggest to explicitly use A and W suffixes to avoid confusion: not just SystemParametersInfo, but SystemParametersInfoA.
For tests, I think we should in fact use SystemParametersInfo, and then compile the test twice -- for A and for W. Both versions should behave the same, right? If we make people write the same code twice (for A and for W), we will either (1) get no tests, or (2) have tests that exercise only one of the two cases. -- Dimi.
--- "Dimitrie O. Paun" <dimi(a)cs.toronto.edu> wrote:
On Thu, 17 Jan 2002, Dmitry Timoshkov wrote:
For tests, I think we should in fact use SystemParametersInfo, and then compile the test twice -- for A and for W. Both versions should behave the same, right?
IMHO (after consulting with MSDN) you can write A<->W portable code only with MFC. Let me know if I'm wrong. Andriy Palamarchuk __________________________________________________ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/
On Thu, 17 Jan 2002, Andriy Palamarchuk wrote:
--- "Dimitrie O. Paun" <dimi(a)cs.toronto.edu> wrote:
On Thu, 17 Jan 2002, Dmitry Timoshkov wrote:
For tests, I think we should in fact use SystemParametersInfo, and then compile the test twice -- for A and for W. Both versions should behave the same, right?
IMHO (after consulting with MSDN) you can write A<->W portable code only with MFC. Let me know if I'm wrong.
You are wrong (or the MSDN is trying to lure you into using the MFC by making false claims). Your code should never use char or WCHAR, strcpy, and things like that. Instead it should use the things defined in 'tchar.h': * TCHAR maps to char or WCHAR * _tcscpy maps to strcpy or wcscpy * _tcslen maps to strlen or wcslen * _TEOF maps to EOF or WEOF * _tWinMain maps to WinMain or wWinMain ... and then the regular structures and APIs: * xxx maps to xxxA or xxxW etc, etc. The problem will be with Winelib's Unicode support though. It has made progress but there may still be some problems and it depends on compiler support. But at least, this will allow us to test it which is a good thing. I believe that C tests should be ANSI/UNICODE independent as much as possible, and then, at compile time, one can choose to compile either for ANSI (default) or for UNICODE. If we can have a test framework that can compile and run a single test twice, once for ANSI and once for UNICODE, then it's perfect. But otherwise the above solution is good enough. Most people will compile and run the test in ANSI mode and some people will compile and run them in Unicode mode, and that's probably good enough. -- Francois Gouget fgouget(a)free.fr http://fgouget.free.fr/ 1 + e ^ ( i * pi ) = 0
Francois Gouget <fgouget(a)free.fr> writes:
I believe that C tests should be ANSI/UNICODE independent as much as possible, and then, at compile time, one can choose to compile either for ANSI (default) or for UNICODE.
No, *please* don't do that. These Unicode macros are an ugliness that we should stay away from; in fact we probably want to make it impossible to use them, just like we do inside Wine itself. Please write tests to call both A and W functions explicitly; it really isn't much more work than testing a single variant anyway. -- Alexandre Julliard julliard(a)winehq.com
On Thu, 17 Jan 2002, Alexandre Julliard wrote:
No, *please* don't do that. These Unicode macros are an ugliness that we should stay away from; in fact we probably want to make it impossible to use them, just like we do inside Wine itself. Please write tests to call both A and W functions explicitly; it really isn't much more work than testing a single variant anyway.
But Alexandre, The Win32 API is meant to be used that way, and so we should test it. Besides, I don't see why they are so ugly. Writing xxx instead of xxxA or xxxW is not ugly in any stretch of the word. Moreover, you have to realize that writing tests is _boring_ code. A test may take 20-30 lines of code or more, asking someone to duplicate all that crap (and maintain, when things break!!!) is simply unreasonable, when it's really dubious what you get out of all that additional effort. Incidentally, I just started working (a few month ago) for a company that's big on XP, and so we write tests for everything we do, so I know what I'm talking about when I say writing tests is a chore! :))) -- Dimi.
"Dimitrie O. Paun" <dimi(a)cs.toronto.edu> writes:
The Win32 API is meant to be used that way, and so we should test it. Besides, I don't see why they are so ugly. Writing xxx instead of xxxA or xxxW is not ugly in any stretch of the word.
What's ugly is that you don't compile what you write. This is the most sure recipe to make sure the code doesn't compile. People will test code in ASCII mode, and when some poor soul (like me ;-) tries to compile in Unicode mode to run regression tests it won't work because nobody ever tried to compile it that way. Or worse it will compile but not do what was intended because someone has been mixing char and tchar or whatever. xxxA and xxxW are different functions (and in fact do not necessarily behave the same way for all inputs), and they need to be tested independently. -- Alexandre Julliard julliard(a)winehq.com
On 17 Jan 2002, Alexandre Julliard wrote: [...]
What's ugly is that you don't compile what you write. This is the most sure recipe to make sure the code doesn't compile. People will test code in ASCII mode, and when some poor soul (like me ;-) tries to compile in Unicode mode to run regression tests it won't work because nobody ever tried to compile it that way. Or worse it will compile but not do what was intended because someone has been mixing char and tchar or whatever.
I would call it part of normal test debugging. You could also reject tests that don't work in both modes. We could also have a Mr 'Unicode' whose work would be to make sure tests work in Unicode and fix them if necessary.
xxxA and xxxW are different functions (and in fact do not necessarily behave the same way for all inputs), and they need to be tested independently.
Most of the time they behave the same way, and in the few cases where they don't, one can still call xxxA and xxxW. And having a single test that can be used to test both A and W reduces code duplication, which helps maintainability and reduces the amount of work we have to do. The only issue I see is if the xxxA is implemented but xxxW is not or reciprocally. Then we have a test that fails but cannot really be put as a TODO since xxxA works. the right way to fix this is to implement the function that is missing. Another way to handle this issue is to also have a 'TODOA' and a 'TODOW'. -- Francois Gouget fgouget(a)free.fr http://fgouget.free.fr/ RFC 2549: ftp://ftp.isi.edu/in-notes/rfc2549.txt IP over Avian Carriers with Quality of Service
Francois Gouget <fgouget(a)free.fr> writes:
And having a single test that can be used to test both A and W reduces code duplication, which helps maintainability and reduces the amount of work we have to do.
...and makes sure that the W functions are never actually tested with Unicode input. It's not enough to simply pass converted ASCII strings to the W functions, we have to test with real Unicode to check for lossy W->A->W conversions, surrogate handling, non-spacing chars, etc. The functions *are* really different and have different testing requirements. -- Alexandre Julliard julliard(a)winehq.com
On Thu, 17 Jan 2002, Alexandre Julliard wrote:
...and makes sure that the W functions are never actually tested with Unicode input. It's not enough to simply pass converted ASCII strings to the W functions, we have to test with real Unicode to check for lossy W->A->W conversions, surrogate handling, non-spacing chars, etc. The functions *are* really different and have different testing requirements.
OK, this is good point. But the truth of the matter is that the semantics of 99.99% of the functions should be encoding independent, because the documentation from MS is, the way you are supposed to write your code is, etc. On the other hand, as you pointed out, we would really want to test all the special handling for the Unicode cases. I think these two requirements will in fact push us to a better sollution. As you well pointed out, when we do write tests, we want to test a lot of things: W->A->W conversions, surrogate handling, non-spacing chars, etc. With the current compiler support, and the current developer knowladge, I am afraid we will only see tests like this: WCHAR teststr[] = { 'U', 'n', 'i', 'c', 'o', 'd', 'e', '\0' }; which tests nothing of the interesting cases you mentioned, and we will endup with twice as much code foolishly. My proposal: -- abstract away the generation of test strings (more of this later) -- write mosts tests in an encoding independent manner -- if explicit A or W handling is required, simply use the xxxA or xxxW without any #ifdefs -- always compile and run the tests two times for ANSI/UNICODE The code will look as such: TCSTR str = TSC_RAND; while ( (str = teststr(str) ) { <your test which uses 'str' goes here> } The TSC_XXX set of constants (TSC stands for Test String Class), identifies a class of strings (such as, 3 charactes long, etc.). The teststr() iterates though those strings. It has the following prototype: TCSTR teststr(TCSTR); And it should be fairly simple to implement (I can provide code, if people are interested). Advantages: -- we can pass in one for the ANSI case, but a few for the Unicode case where we want to test various things. -- for regular unit testing, we may want to return only one representative string for ANSI, and one complicated string for Unicode, but from time to time we may want to run the tests over a large number of test strings. -- the tests so writen can be used to identify and test performance problems, by simply returning a lot more strings, and by logging how long it took to run the test. -- we have a clean way to abstract the testing string, and so most of the test can be written in an encoding independent way. -- it makes it no harder, in _any_ way, to write encoding specific tests. In fact, one can use the teststrA/teststrW versions for their encoding specific tests to get the above mentioned benefits. -- it should be trivial to instrument the Makefiles to compile the tests twice, and thus we should test both the ANSII and the Unicode cases all the time. (that is, this should not be an Alexandre only task :)) Comments? -- Dimi.
--- "Dimitrie O. Paun" <dimi(a)cs.toronto.edu> wrote:
On Thu, 17 Jan 2002, Alexandre Julliard wrote:
-- if explicit A or W handling is required, simply use the xxxA or xxxW without any #ifdefs
You definitely need #ifdefs or just unicode conditional block around explicit calls of xxxW functions. Otherwise these functions will be called on platforms which do not support Unicode, even if test application is compiled in ANSI mode. In general I like your idea. Andriy Palamarchuk __________________________________________________ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/
On Fri, 18 Jan 2002, Andriy Palamarchuk wrote:
You definitely need #ifdefs or just unicode conditional block around explicit calls of xxxW functions. Otherwise these functions will be called on platforms which do not support Unicode, even if test application is compiled in ANSI mode.
Fine, you have a point. Just do: if (UNICODE) { <W explicit code> } instead. I hate #ifdefs! If we do that, we can use the tests to also determine the performance difference between A & W, which is great 'cause it will give us an idea of the penalty for all those string translations and memory allocations. (I think this can be added to the 'Advantages' section of my previous mail, :) )
In general I like your idea.
Thanks. -- Dimi.
Francois Gouget wrote:
The only issue I see is if the xxxA is implemented but xxxW is not or reciprocally. Then we have a test that fails but cannot really be put as a TODO since xxxA works. the right way to fix this is to implement the function that is missing. Another way to handle this issue is to also have a 'TODOA' and a 'TODOW'.
Oh, that's an easy one :-) : #ifdef _UNICODE TODO(xxxW) #else xxxA #endif Andriy Palamarchuk
Andriy Palamarchuk <apa3a(a)yahoo.com> writes:
Can you suggest more reliable way to detect if test runs on Wine?
I'll work on that.
One more important change in C framework is support of Unicode in testing. Dimitry, can you look at it? See how it is used in sysparams.c, search for "has_unicode" function call. I'm interested in your comments.
I don't think we want that. And anyway we don't emulate non-Unicode platform properly (i.e. Unicode calls that fail under win95 will succeed under Wine even with -winver win95). I think we should simply call Unicode functions on all platforms and have the tests deal with the possible errors. -- Alexandre Julliard julliard(a)winehq.com
participants (5)
-
Alexandre Julliard -
Andriy Palamarchuk -
Dimitrie O. Paun -
Dmitry Timoshkov -
Francois Gouget