Hi,
Forgot to include wine-devel.
-------- Original Message -------- Subject: [winetest] New gather script
Hi Ferenc,
Could you have a look at the new 'gather' script? (you're the original author after all).
The new winetest (when committed of course) executable will report when a test is missing because of a failing dll.
I've severely changed/mutilated 'gather' so:
- it finds these missing dll tests and marks them - doesn't add these to the overall list of tests - goes through the whole list of tests to find out if (and why) tests are missing - it now shows also complete failing tests (for the group) in the group overview - add some extra onMouseOver on both group and single tests. - make sure we always show missing tests in the group overview.
The new variables:
$test->{dllmissing}->{$dll} (shows if a system/test is missing a dll)
$group->{dllmissing}->{$unit} (tells if at least one system in the group is missing this dll, used for having a nice onMouseOver in the Main Summary)
$group->{testmissing}->{$testname} (tells if at least one system in the group has a missing test for an unknown reason, needed to cope with the same onMouseOver as mentioned above).
I'll split it up in smaller chunks when sent to wine-patches.
Cheers,
Paul.
Index: gather =================================================================== RCS file: /home/wine/tools/winetest/gather,v retrieving revision 1.20 diff -u -r1.20 gather --- gather 13 Mar 2007 16:10:46 -0000 1.20 +++ gather 3 Aug 2007 07:28:36 -0000 @@ -96,9 +96,19 @@ my ($digest, $unit, $test, $count, $todo, $error, $skipped, $source, $rev) = split; my $testname = "$unit:$test"; + if ($test =~ /_dll_missing/) { + # Mark this dll as being missing on one or more systems + + # For this run on this system + $testref->{dllmissing}->{$unit} = "This dll is missing on this system"; + # For the whole group + $group->{dllmissing}->{$unit} = "This dll is missing for some tests"; + } + else { + $alltests{$testname} = "http://cvs.winehq.org/cvsweb/wine/$source" . ($rev ne "-"?"#rev$rev":"") + unless exists $alltests{$testname}; + } $testref->{results}->{$testname} = [$count, $todo, $error, $skipped]; - $alltests{$testname} = "http://cvs.winehq.org/cvsweb/wine/$source" . ($rev ne "-"?"#rev$rev":"") - unless exists $alltests{$testname}; if ($count ne "failed") { if (defined $group->{extrema}->{$testname}) { my $extrema = $group->{extrema}->{$testname}; @@ -117,6 +127,28 @@ push @{$group->{tests}}, $testref; }
+# Find missing tests and/or missing dlls and mark them +foreach my $group (@groups) { + next unless exists $group->{tests}; + foreach my $testname (sort keys %alltests) { + foreach my $test (@{$group->{tests}}) { + if (!exists $test->{results}->{$testname}) { + # The test is missing, try to find the reason + my ($dll,$subtest) = split(/:/, $testname); + if (exists $test->{dllmissing}->{$dll}) { + $test->{results}->{$testname} = [-1, -1, -1, -1]; + $group->{digest}->{$testname} = "differ"; + } + else { + $test->{results}->{$testname} = [-2, -2, -2, -2]; + $group->{digests}->{$testname} = "differ"; + $group->{testmissing}->{$testname} = $testname; + } + } + } + } +} + # Write out the tables
my ($header); # same in thead and tfoot @@ -197,28 +229,42 @@ # Output a single cell of a test sub singletest { my ($test, $testname, $groupname) = @_; - - if (!exists $test->{results}->{$testname}) { - print OUT " <td class="note">.</td>\n"; + my $file = "$test->{dir}/$testname.txt"; + my ($count, $todo, $error, $skipped) = @{$test->{results}->{$testname}}; + if ($count eq "failed") { + # test run failed + my $msg = $todo eq "-"?"crash": + $todo == 258?"timeout":"failed"; + my $fail = -r "$datadir/$build/$file"?"<a href="$file">$msg</a>":$msg; + print OUT " <td class="note">$fail</td>\n"; + } elsif ($count == -1) { + # dll missing + my ($dll, $subtest) = split(/:/, $testname); + print OUT <<"EOF"; + <td class="skip_pass"><a + title="No tests run as $dll.dll is not present on this system" + onMouseOver="No tests run as $dll.dll is not present on this system" + >0</a></td> +EOF + } elsif ($count == -2) { + # test missing + print OUT <<"EOF"; + <td class="skip_fail"><a + title="No tests run for an unknown reason" + onMouseOver="No tests run for an unknown reason" + >0</a></td> +EOF } else { - my $file = "$test->{dir}/$testname.txt"; - my ($count, $todo, $error, $skipped) = @{$test->{results}->{$testname}}; - if ($count eq "failed") { - my $msg = $todo eq "-"?"crash": - $todo == 258?"timeout":"failed"; - my $fail = -r "$datadir/$build/$file"?"<a href="$file">$msg</a>":$msg; - print OUT " <td class="note">$fail</td>\n"; - } else { - my $class = $error?"fail":"pass"; - my $skip = $skipped?"skip_":""; - print OUT <<"EOF"; + # test has run + my $class = $error?"fail":"pass"; + my $skip = $skipped?"skip_":""; + print OUT <<"EOF"; <td class="$skip$class"><a href="$file" title="$count tests, $todo TODO, $error errors, $skipped skipped" onMouseOver="refresh('$testname','$groupname $test->{tag}',$count,$todo,$error,$skipped);" >$error</a></td> EOF - } } }
@@ -236,6 +282,7 @@ } else { my $href = "href="#$group->{name}""; if (exists $group->{extrema}->{$testname}) { + # At least one system in this group has run this test. my ($min,$max) = @{$group->{extrema}->{$testname}}; my $class = $min==0?($max==0?"pass":"mixed"):"fail"; my $title = $min==0?"":" title="Best: $min""; @@ -246,7 +293,23 @@ >$max</a></td> EOF } else { - print OUT " <td class="note"><a $href>.</a></td>\n"; + my ($dll, $subtest) = split(/:/, $testname); + if ($group->{testmissing}->{$testname}) { + # At least one system in the group has a missing test (but + # not because of a missing dll). + print OUT " <td class="fail"><a $href>.</a></td>\n"; + } elsif (exists $group->{dllmissing}->{$dll}) { + # All systems in the group have a missing dll related to this test. + print OUT <<"EOF"; + <td class="pass"><a $href + title="No tests run as $dll.dll is not present on all $group->{name} systems" + onMouseOver="No tests run as $dll.dll is not present on all $group->{name} systems" + >0</a></td> +EOF + } else { + # All systems in this group failed to correctly run the test. + print OUT " <td class="fail"><a $href>.</a></td>\n"; + } } } }