Cmd tests timeout and splitting tests

[Octavian Voicu]

Hello,

I've noticed a lot of timeouts for cmd tests lately, which is not so
surprising with all the new incoming tests.

I think test_builtins.cmd should be split in a few files, which would
then be run separately. An added benefit is that it would permit
putting tests that affect other tests in separate files.

In order to fix the timeout problem it's not enough to split the tests
in different files. Each file has to be run on its own from winetest.

I see following solutions:

1) Hack winetest and increase timeout just for cmd.exe batch tests.

2) Allow individual tests to receive a parameter.

Downside is that make_ctests needs to be changed a bit (to recognize
filenames given as "test.c:param"), and a manual rule for running
MAKECTESTS needs to be added to the Makefile of those tests, which
would list all the tests + values for the params.

3) Add a new source file for each .cmd file.

All the batch running meat would still be in batch.c, with main test
function non-static. A header file with prototypes and a macro for
defining a test would be included in subsequent tests.

I would go for number 3, I think it's the cleanest solution (although
it would add a few more source files). Any comments on that?

Octavian

[Frédéric Delanoy]

On Sun, Aug 28, 2011 at 01:24, Octavian Voicu  wrote:
> Hello,
>
> I've noticed a lot of timeouts for cmd tests lately, which is not so
> surprising with all the new incoming tests.

Which timeouts are you talking about? On windows machines? Never
experienced a timeout
Could you give me some link?
> I think test_builtins.cmd should be split in a few files, which would
> then be run separately. An added benefit is that it would permit
> putting tests that affect other tests in separate files.

It's difficult to test in complete isolation given that we use our own
dogfood (i.e. run with cmd) for cmd tests.
Also you would have to run all the tests anyway, since a small change
to a builtin can lead to regressions in other parts.
The association .cmd/.exp would have to be handled specially as well.

Frédéric

[Alexandre Julliard]

Frédéric Delanoy  writes:
> On Sun, Aug 28, 2011 at 01:24, Octavian Voicu  wrote:
> > Hello,
> >
> > I've noticed a lot of timeouts for cmd tests lately, which is not so
> > surprising with all the new incoming tests.
>
> Which timeouts are you talking about? On windows machines? Never
> experienced a timeout

It only times out on Wine, because cmd is too slow (reading the input
char by char and other silliness). That's what should be fixed.

[Frédéric Delanoy]

On Sun, Aug 28, 2011 at 01:24, Octavian Voicu  wrote:
> Hello,
>
> I've noticed a lot of timeouts for cmd tests lately, which is not so
> surprising with all the new incoming tests.

OK found them
> I think test_builtins.cmd should be split in a few files, which would
> then be run separately. An added benefit is that it would permit
> putting tests that affect other tests in separate files.
>
> In order to fix the timeout problem it's not enough to split the tests
> in different files. Each file has to be run on its own from winetest.
>
> I see following solutions:
> -
>
> Hack winetest and increase timeout just for cmd.exe batch tests.
> -
>
> Allow individual tests to receive a parameter.
>
> <...>
> - Add a new source file for each .cmd file.
>
> <...>
> I would go for number 3, I think it's the cleanest solution (although
> it would add a few more source files). Any comments on that?

1) would probably need to be done temporarily until a proper solution
is implemented

Frédéric