Unknown issue with test.winehq.org

[Paul Vriens]

Hi,

So finally we have a working winetest executable and the test.winehq.org scripts 
should be able to handle the new reports.

Could someone have a look what the issue currently is on the webserver as I've 
sent several reports (a few hours ago) and they are not handled yet?

Cheers,

Paul.

[Paul Vriens]

Paul Vriens wrote:
> Hi,
>
> So finally we have a working winetest executable and the test.winehq.org 
> scripts should be able to handle the new reports.
>
> Could someone have a look what the issue currently is on the webserver 
> as I've sent several reports (a few hours ago) and they are not handled 
> yet?
>
> Cheers,
>
> Paul.

Hi,

I think I've found the issue.

The error dissect outputs is something like:

/wine/winetest/queue/errAfg8T: no start line:4789: ntdll:exception done (5)

when looking at the report file I see that with a newer version of the ntdll 
exception tests we suddenly have 2 lines:

exception: 42 tests executed (0 marked as todo, 0 failures), 0 skipped.
exception: 279 tests executed (0 marked as todo, 5 failures), 0 skipped.

The first one triggers the 'end of this test' in dissect. The second one is thus 
not accepted as dissect expects a 'start' line (and a line in between actually).

The question now is: should dissect handle these cases or is this particular 
test-case wrong?

Cheers,

Paul.

[Alexandre Julliard]

Paul Vriens <paul.vriens.wine(a)gmail.com> writes:
> exception: 42 tests executed (0 marked as todo, 0 failures), 0 skipped.
> exception: 279 tests executed (0 marked as todo, 5 failures), 0 skipped.
>
> The first one triggers the 'end of this test' in dissect. The second
> one is thus not accepted as dissect expects a 'start' line (and a line
> in between actually).
>
> The question now is: should dissect handle these cases or is this
> particular test-case wrong?

It's because some of the tests are run in a child process. dissect
should probably handle it, ideally by reporting the sum of the test
counts as the results for the whole test.

-- 
Alexandre Julliard
julliard(a)winehq.org

[Paul Vriens]

Alexandre Julliard wrote:
> Paul Vriens <paul.vriens.wine(a)gmail.com> writes:
> > exception: 42 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > exception: 279 tests executed (0 marked as todo, 5 failures), 0 skipped.
> >
> > The first one triggers the 'end of this test' in dissect. The second
> > one is thus not accepted as dissect expects a 'start' line (and a line
> > in between actually).
> >
> > The question now is: should dissect handle these cases or is this
> > particular test-case wrong?
>
> It's because some of the tests are run in a child process. dissect
> should probably handle it, ideally by reporting the sum of the test
> counts as the results for the whole test.

So what's the difference between this one and for example the following ones 
running in child processes:

kernel32:process start dlls/kernel32/tests/process.c 1.10
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
process: 396 tests executed (0 marked as todo, 6 failures), 0 skipped.

or:

msvcrt:file start dlls/msvcrt/tests/file.c 1.45
tests/file.c: 3 tests executed (0 marked as todo, 0 failures), 0 skipped.
tests/file.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
file: 361 tests executed (0 marked as todo, 0 failures), 0 skipped.

or better yet:

shell32:shellpath start dlls/shell32/tests/shellpath.c 1.16
tests/shellpath.c: 10 tests executed (0 marked as todo, 4 failures), 0 skipped.
tests/shellpath.c: 4 tests executed (0 marked as todo, 0 failures), 0 skipped.
shellpath: 452 tests executed (0 marked as todo, 0 failures), 0 skipped.

Obviously the start of line is a filename for these ones, whereas the start of 
the line for the exception one is the test itself.

Should we consolidate the above ones as well? I think yes, cause the shell32 
example is reflected as 0 errors on test.winehq.org.

Cheers,

Paul.

[Peter Oberndorfer]

On Thursday 01 March 2007 08:30, Paul Vriens wrote:
> Alexandre Julliard wrote:
> > Paul Vriens <paul.vriens.wine(a)gmail.com> writes:
> > > exception: 42 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > > exception: 279 tests executed (0 marked as todo, 5 failures), 0 skipped.
> > >
> > > The first one triggers the 'end of this test' in dissect. The second
> > > one is thus not accepted as dissect expects a 'start' line (and a line
> > > in between actually).
> > >
> > > The question now is: should dissect handle these cases or is this
> > > particular test-case wrong?
> >
> > It's because some of the tests are run in a child process. dissect
> > should probably handle it, ideally by reporting the sum of the test
> > counts as the results for the whole test.
>
> So what's the difference between this one and for example the following ones 
> running in child processes:
>
> kernel32:process start dlls/kernel32/tests/process.c 1.10
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> process: 396 tests executed (0 marked as todo, 6 failures), 0 skipped.
>
> or:
>
> msvcrt:file start dlls/msvcrt/tests/file.c 1.45
> tests/file.c: 3 tests executed (0 marked as todo, 0 failures), 0 skipped.
> tests/file.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> file: 361 tests executed (0 marked as todo, 0 failures), 0 skipped.
>
> or better yet:
>
> shell32:shellpath start dlls/shell32/tests/shellpath.c 1.16
> tests/shellpath.c: 10 tests executed (0 marked as todo, 4 failures), 0 skipped.
> tests/shellpath.c: 4 tests executed (0 marked as todo, 0 failures), 0 skipped.
> shellpath: 452 tests executed (0 marked as todo, 0 failures), 0 skipped.
>
> Obviously the start of line is a filename for these ones, whereas the start of 
> the line for the exception one is the test itself.
>
> Should we consolidate the above ones as well? I think yes, cause the shell32 
> example is reflected as 0 errors on test.winehq.org.
>
> Cheers,
>
> Paul.

Hi,
sorry for not replying earlier, i didn't check mail yesterday.

If the problem is only the prefix of the test result line the following trivial patch will make the tests work again.
Output is now:

... more traces....
exception.c:644:excpetion 0xc0000008 at 00000000 firstchance=0 Eip=0x24000b, Eax=0xf00f00f0
tests/exception.c: 42 tests executed (22 marked as todo, 0 failures), 0 skipped.
exception: 280 tests executed (13 marked as todo, 0 failures), 0 skipped.

Will send to wine-patches.
Sorry for breaking tests :-/

Greetings Peter

[Paul Vriens]

Peter Oberndorfer wrote:
> On Thursday 01 March 2007 08:30, Paul Vriens wrote:
> > Alexandre Julliard wrote:
> > > Paul Vriens <paul.vriens.wine(a)gmail.com> writes:
> > > > exception: 42 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > > > exception: 279 tests executed (0 marked as todo, 5 failures), 0 skipped.
> > > >
> > > > The first one triggers the 'end of this test' in dissect. The second
> > > > one is thus not accepted as dissect expects a 'start' line (and a line
> > > > in between actually).
> > > >
> > > > The question now is: should dissect handle these cases or is this
> > > > particular test-case wrong?
> > > > It's because some of the tests are run in a child process. dissect
> > > > should probably handle it, ideally by reporting the sum of the test
> > > > counts as the results for the whole test.
> >
> > So what's the difference between this one and for example the following ones 
> > running in child processes:
> >
> > kernel32:process start dlls/kernel32/tests/process.c 1.10
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/process.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > process: 396 tests executed (0 marked as todo, 6 failures), 0 skipped.
> >
> > or:
> >
> > msvcrt:file start dlls/msvcrt/tests/file.c 1.45
> > tests/file.c: 3 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > tests/file.c: 1 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > file: 361 tests executed (0 marked as todo, 0 failures), 0 skipped.
> >
> > or better yet:
> >
> > shell32:shellpath start dlls/shell32/tests/shellpath.c 1.16
> > tests/shellpath.c: 10 tests executed (0 marked as todo, 4 failures), 0 skipped.
> > tests/shellpath.c: 4 tests executed (0 marked as todo, 0 failures), 0 skipped.
> > shellpath: 452 tests executed (0 marked as todo, 0 failures), 0 skipped.
> >
> > Obviously the start of line is a filename for these ones, whereas the start of 
> > the line for the exception one is the test itself.
> >
> > Should we consolidate the above ones as well? I think yes, cause the shell32 
> > example is reflected as 0 errors on test.winehq.org.
> >
> > Cheers,
> >
> > Paul.
>
> Hi,
> sorry for not replying earlier, i didn't check mail yesterday.
>
> If the problem is only the prefix of the test result line the following trivial patch will make the tests work again.
> Output is now:
>
> ... more traces....
> exception.c:644:excpetion 0xc0000008 at 00000000 firstchance=0 Eip=0x24000b, Eax=0xf00f00f0
> tests/exception.c: 42 tests executed (22 marked as todo, 0 failures), 0 skipped.
> exception: 280 tests executed (13 marked as todo, 0 failures), 0 skipped.
>
> Will send to wine-patches.
> Sorry for breaking tests :-/
>
> Greetings Peter
>
> ------------------------------------------------------------------------
>
> diff --git a/dlls/ntdll/tests/exception.c b/dlls/ntdll/tests/exception.c
> index 08ec9c6..9a70264 100644
> --- a/dlls/ntdll/tests/exception.c
> +++ b/dlls/ntdll/tests/exception.c
> @@ -600,7 +600,7 @@ static void test_debugger(void)
>          return;
>      }
>
> -    sprintf(cmdline, "%s %s %s", my_argv[0], my_argv[1], "debuggee");
> +    sprintf(cmdline, "%s %s %s", my_argv[0], "tests/exception.c", "debuggee");
>      ok(CreateProcess(NULL, cmdline, NULL, NULL, FALSE, DEBUG_PROCESS, NULL, NULL,
>                       &si, &pi) != 0, "error: %u\n", GetLastError());

Hi,

I actually sent in a patch (for the website) earlier today that should deal with 
this as well. My patch  makes it possible to have both types of result-lines and 
the number will be added for every single test (this wasn't the case before).

Don't know if both should go in? We definitely need mine (or something alike if 
Jeremy, or anybody for that matter, doesn't like it) to cope with these multiple 
lines.

Cheers,

Paul.