I am implementing my tests as follows:
* patch 1: general test for a D3DX function (D3DXCreateBox for example, testing specific vertices and indices for _specific_ dimensions). * patch 2: separate into function that works with arbitrary dimensions for the box, etc. I restrict to the dimensions from patch 1 using assert statements. * patch 3: expand to allow arbitrary parameters to the function, e.g., arbitrary dimensions, removing the assert statements.
For example, see
patch 2 = http://github.com/misha680/wine/commit/6291ca28593c78761ccef3137b1014bae87d2...
patch 3 = http://github.com/misha680/wine/commit/7b09cfb661bf1d03fd80195c0cd2b7b3bb9f1...
Is assert the right way to do this? If not, what is?
Thank you Misha
On 10 July 2010 03:40, Misha Koshelev misha680@gmail.com wrote:
I am implementing my tests as follows:
- patch 1: general test for a D3DX function (D3DXCreateBox for example,
testing specific vertices and indices for _specific_ dimensions).
- patch 2: separate into function that works with arbitrary dimensions
for the box, etc. I restrict to the dimensions from patch 1 using assert statements.
- patch 3: expand to allow arbitrary parameters to the function, e.g.,
arbitrary dimensions, removing the assert statements.
For example, see
patch 2 = http://github.com/misha680/wine/commit/6291ca28593c78761ccef3137b1014bae87d2...
patch 3 = http://github.com/misha680/wine/commit/7b09cfb661bf1d03fd80195c0cd2b7b3bb9f1...
Is assert the right way to do this? If not, what is?
Assert should be avoided in the tests.
If the assert fails, you will see that the test crashed -- especially when the results are on tests.winehq.org. This makes it difficult to track what went wrong.
You should use an ok or skip/win_skip message so that information gets logged and people looking at the failure can see what failed.
- Reece
On Sat, 2010-07-10 at 07:40 +0100, Reece Dunn wrote:
On 10 July 2010 03:40, Misha Koshelev misha680@gmail.com wrote:
I am implementing my tests as follows:
- patch 1: general test for a D3DX function (D3DXCreateBox for example,
testing specific vertices and indices for _specific_ dimensions).
- patch 2: separate into function that works with arbitrary dimensions
for the box, etc. I restrict to the dimensions from patch 1 using assert statements.
- patch 3: expand to allow arbitrary parameters to the function, e.g.,
arbitrary dimensions, removing the assert statements.
For example, see
patch 2 = http://github.com/misha680/wine/commit/6291ca28593c78761ccef3137b1014bae87d2...
patch 3 = http://github.com/misha680/wine/commit/7b09cfb661bf1d03fd80195c0cd2b7b3bb9f1...
Is assert the right way to do this? If not, what is?
Assert should be avoided in the tests.
If the assert fails, you will see that the test crashed -- especially when the results are on tests.winehq.org. This makes it difficult to track what went wrong.
You should use an ok or skip/win_skip message so that information gets logged and people looking at the failure can see what failed.
- Reece
Ok that makes sense.
What about in the case of something like this:
/* http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm * "I think the answer is 10,000 but since floating point math is imperfect I’ll accept the maxUlps floats above and the maxUlps floats below that value." */ static BOOLEAN AlmostEqual2sComplement(float A, float B, int maxUlps) { int aInt, bInt, intDiff; /* Make sure maxUlps is non-negative and small enough that the * default NAN won't compare as equal to anything. */ assert(maxUlps > 0 && maxUlps < 4 * 1024 * 1024); aInt= *(int*)&A; /* Make aInt lexicographically ordered as a twos-complement int */ if (aInt < 0) aInt = 0x80000000 - aInt; /* Make bInt lexicographically ordered as a twos-complement int */ bInt = *(int*)&B; if (bInt < 0) bInt = 0x80000000 - bInt; intDiff = abs(aInt - bInt); if (intDiff <= maxUlps) return TRUE; return FALSE; }
Is assert ok here or do I need to change it to some kind of skip statement as well?
The problem here is that the skip statement would be inside a helper function, and I guess it wouldn't actually skip.
I could always return FALSE, but that doesn't seem quite right here.
Should I just take the assert out altogether here then if they are not allowed in tests or is it not so black & white?
Thanks Misha
On 10 July 2010 17:40, Misha Koshelev misha680@gmail.com wrote:
On Sat, 2010-07-10 at 07:40 +0100, Reece Dunn wrote:
On 10 July 2010 03:40, Misha Koshelev misha680@gmail.com wrote:
Ok that makes sense.
What about in the case of something like this:
/* http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm * "I think the answer is 10,000 but since floating point math is imperfect I’ll accept the maxUlps floats above and the maxUlps floats below that value." */ static BOOLEAN AlmostEqual2sComplement(float A, float B, int maxUlps) { int aInt, bInt, intDiff; /* Make sure maxUlps is non-negative and small enough that the * default NAN won't compare as equal to anything. */ assert(maxUlps > 0 && maxUlps < 4 * 1024 * 1024);
Is assert ok here or do I need to change it to some kind of skip statement as well?
The assert should be ok here provided that maxUlps does not come from some function -- that is, it is constant and will trigger on all systems.
Assert statements are not forbidden in tests (there are some already in various tests), it's just that if they trigger as a result of the system as part of calls under test, they make it difficult to track down what is failing and why by looking at the test results. If the asserts are triggered on any system because they are the result of programmer error (e.g. typo) should be fine.
- Reece
Thank you for the explanation. maxUlps is in fact a constant.
Although the asserts in my previous question re incremental patches were also only applicable due to programmer error. However, I think in that case a skip is much more appropriate if nothing else because it is immediately clear the programmer made an error.
Thank you Misha
On Jul 10, 2010 12:11 PM, "Reece Dunn" msclrhd@googlemail.com wrote:
On 10 July 2010 17:40, Misha Koshelev misha680@gmail.com wrote:
On Sat, 2010-07-10 at 07:40 +010...
Ok that makes sense.
What about in the case of something like this:
/* http://www.cygnus...
Is assert ok here or do I need to change it to some kind of skip statement as well?
The assert should be ok here provided that maxUlps does not come from some function -- that is, it is constant and will trigger on all systems.
Assert statements are not forbidden in tests (there are some already in various tests), it's just that if they trigger as a result of the system as part of calls under test, they make it difficult to track down what is failing and why by looking at the test results. If the asserts are triggered on any system because they are the result of programmer error (e.g. typo) should be fine.
- Reece
Just for thoroughness and clarity... I would like to put BOTH a skip that outputs that test was skipped and assert that then keeps any further damage being done. This is in the case of maxUlps. Any objections?
Thanks Misha
On Jul 10, 2010 12:11 PM, "Reece Dunn" msclrhd@googlemail.com wrote:
On 10 July 2010 17:40, Misha Koshelev misha680@gmail.com wrote: On Sat, 2010-07-10 at 07:40 +010...
Ok that makes sense.
What about in the case of something like this:
/*
Is assert ok here or do I need to change it to some kind of skip statement as well?
T...
-
Misha Koshelev -
Reece Dunn