Re: [PATCH 5] winmm/tests: Add MCI waveaudio tests.

[Alexandre Julliard]

 writes:
> -START_TEST(mci)
> +#ifndef FULL_TEST
> +/* Some tests hang or crash Wine. Enable once fixed. */
> +#define FULL_TEST 0
> +#endif

Please don't introduce dead code. If the tests really can't work then
don't add them at all until you have the corresponding fixes.

[PATCH 5] winmm/tests: Add MCI waveaudio tests.

Hi,
> +/* Some tests hang or crash Wine. Enable once fixed. */
> +#define FULL_TEST 0
> Please don't introduce dead code. If the tests really can't work then
> don't add them at all until you have the corresponding fixes.

Will do.  Although I'd very much prefer having tests than no tests.
I've been fixing MCI regressions that were as old as 2001 that would
hopefully not have occured if there had been tests in the first place.
The #if FULL_TEST is very useful when compiling on MS-W* to immediately
spot what Wine should eventually do, and as direction for others
when I'll be gone.

How do other people deal with that situation?
 - use some patch manager, e.g. quilt?
 - maintain seprate test files for running under MS-W*?
 - ignore, write no tests and wait for bugs.winehq?
 - ...?

Regards,
 Jörg Höhle
PS: despite my 20 patches to MCI, there's still need for #if !FULL_TEST.

[PATCH 5] winmm/tests: Add MCI waveaudio tests.

[Reece Dunn]

2009/10/26  :
> Hi,
> > +/* Some tests hang or crash Wine. Enable once fixed. */
> > +#define FULL_TEST 0
> > Please don't introduce dead code. If the tests really can't work then
> > don't add them at all until you have the corresponding fixes.
>
> Will do.  Although I'd very much prefer having tests than no tests.
> I've been fixing MCI regressions that were as old as 2001 that would
> hopefully not have occured if there had been tests in the first place.
> The #if FULL_TEST is very useful when compiling on MS-W* to immediately
> spot what Wine should eventually do, and as direction for others
> when I'll be gone.
>
> How do other people deal with that situation?
>  - use some patch manager, e.g. quilt?
>  - maintain seprate test files for running under MS-W*?
>  - ignore, write no tests and wait for bugs.winehq?
>  - ...?

If the tests are returning incorrect results in Wine, the practice
when writing the tests is to add todo_wine before the ok statement.
Once the fix has been added, the todo_wine bit can be removed. E.g.

before fix:

todo_wine
    ok(hr == S_OK, "should return S_OK, got: %08x", hr);

after fix:

ok(hr == S_OK, "should return S_OK, got: %08x", hr);

Alternatively, you can fix a Wine bug and add the test for it in the
same patch (or with the tests added in the next patch in the series).
These should ideally be tested on Wine and at least one Windows
platform. Other platforms will be picked up on the tests.winehq.org
site.

- Reece

[PATCH 5] winmm/tests: Add MCI waveaudio tests.

Reece dunn wrote:
> Once the fix has been added, the todo_wine bit can be removed.

I know about it and skip and win_skip thanks to Michael Karcher's excellent text:
http://www.winehq.org/pipermail/wine-devel/2008-October/069721.html

The problem is that running the full tests crashes or hangs Wine,
but not MS-Windows platforms.  A hang or crash yields less useable results
on test.winehq and is not nice to users.
> Other platforms will be picked up on the tests.winehq.org site.

What I'm missing is sort of the converse of skip:
Perform a test on all platforms *except* Wine, such that I can later
observe on test.winehq the results of a variety of original MS-Windows systems
- many more than the few I can access.

How to achieve this?

Thanks,
    Jörg Höhle

[PATCH 5] winmm/tests: Add MCI waveaudio tests.

[Paul Vriens]

On 10/27/2009 10:13 AM, [e-mail] wrote:
> Reece dunn wrote:
> > Once the fix has been added, the todo_wine bit can be removed.
>
> I know about it and skip and win_skip thanks to Michael Karcher's excellent text:
> http://www.winehq.org/pipermail/wine-devel/2008-October/069721.html
>
> The problem is that running the full tests crashes or hangs Wine,
> but not MS-Windows platforms.  A hang or crash yields less useable results
> on test.winehq and is not nice to users.
> > Other platforms will be picked up on the tests.winehq.org site.
>
> What I'm missing is sort of the converse of skip:
> Perform a test on all platforms *except* Wine, such that I can later
> observe on test.winehq the results of a variety of original MS-Windows systems
> - many more than the few I can access.
>
> How to achieve this?
>
> Thanks,
>    Jörg Höhle

Isn't there anything in your current test suite that succeeds on Windows 
but fails on Wine? This could be as simple as returning a different last 
error, or not exporting a function (a one not used yet, of course).

[PATCH 5] winmm/tests: Add MCI waveaudio tests.

Paul Virnes wrote:
> > What I'm missing is sort of the converse of skip:
> > Perform a test on all platforms *except* Wine
>
> Isn't there anything in your current test suite that succeeds on Windows
> but fails on Wine?

This is it!
So simple, right in front of my eyes.
Thanks,
  Jörg Höhle.