[tools 1/5] testbot/TestWTBS: Skip the base Windows VMs if there is no base build VM.

[Francois Gouget]

Do not report errors about missing Windows tasks if the TestBot only has
Wine VMs.

Signed-off-by: Francois Gouget <fgouget(a)codeweavers.com>
---
 testbot/tests/TestWTBS | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/testbot/tests/TestWTBS b/testbot/tests/TestWTBS
index 5f62d634e..c5ac9e1a4 100755
--- a/testbot/tests/TestWTBS
+++ b/testbot/tests/TestWTBS
@@ -696,6 +696,12 @@ foreach my $VM (@{$VMs->GetItems()})
 {
   $HasBaseVM->{$VM->Type} = 1 if ($VM->Role eq "base");
 }
+if (!$HasBaseVM->{build})
+{
+  # Cannot run the tests on Windows if there is no build VM
+  delete $HasBaseVM->{win32};
+  delete $HasBaseVM->{win64};
+}
 
 CheckJobs();
 
-- 
2.20.1

[tools 2/5] testbot/TestWTBS: Tweak the properties propagation.

[Francois Gouget]

Cascade the properties down 1 level at a time rather than straight
to the base level. This reduces duplication about which base categories
belong in the top-level ones.

Signed-off-by: Francois Gouget <fgouget(a)codeweavers.com>
---
 testbot/tests/TestWTBS | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/testbot/tests/TestWTBS b/testbot/tests/TestWTBS
index c5ac9e1a4..5cadc5df5 100755
--- a/testbot/tests/TestWTBS
+++ b/testbot/tests/TestWTBS
@@ -323,9 +323,9 @@ sub LoadTestInfo($)
   SetDefault($TestInfo, "tasks", "HasTask", 1);
 
   # Then propagate the defaults
-  foreach my $Pair (["win", ["win32", "win64"]],
-                    ["tests", ["win32", "win64", "wine"]],
-                    ["tasks", ["build", "win32", "win64", "wine"]])
+  foreach my $Pair (["tasks", ["build", "tests"]],
+                    ["tests", ["win", "wine"]],
+                    ["win", ["win32", "win64"]])
   {
     my ($Src, $TaskTypes) = @$Pair;
     foreach my $Field (keys %{$TestInfo->{$Src}})
-- 
2.20.1

[tools 3/5] testbot/TestWTBS: Allow matching errors in testbot.log.

[Francois Gouget]

Tasks have three types of logs:
- *.report files that collect the test output.
- task.log which collects the messages from the VM-side process that
  starts the Wine test / build.
- testbot.log which collects the server-side messages when some
  inconsistency is detected or the connection to the VM is lost.
Each get their own .errors file but so far only errors in the first two
types of logs could be matched.

Signed-off-by: Francois Gouget <fgouget(a)codeweavers.com>
---
 testbot/tests/TestWTBS | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/testbot/tests/TestWTBS b/testbot/tests/TestWTBS
index 5cadc5df5..e3909a22d 100755
--- a/testbot/tests/TestWTBS
+++ b/testbot/tests/TestWTBS
@@ -300,7 +300,7 @@ sub LoadTestInfo($)
   foreach my $RawGroupName (@{$RawInfo->{ErrGroupNames}})
   {
     my $GroupName = lcfirst($RawGroupName);
-    if ($GroupName =~ s/^(tasks|build|tests|win|win32|win64|wine)\.(log|report)\.//)
+    if ($GroupName =~ s/^(tasks|build|tests|win|win32|win64|wine)\.(report|log|testbot)\.//)
     {
       my $ErrInfo = ($TestInfo->{$1}->{"$2.errors"} ||= {});
       push @{$ErrInfo->{ErrGroupNames}}, $GroupName;
@@ -395,12 +395,13 @@ sub IsMailingListJob($)
 }
 
 =pod
-=item <tasks.(log|report).groupname>
+=item <tasks.(report|log|testbot).groupname>
 
 Verifies the presence of new errors in the specified error log or report
 of the tasks in the specified category. For this, list the group and errors
 that are expected to appear; where the group name is prefixed by the task
-category and either 'log' or 'report'.
+category and either 'report' for the test reports, 'log' for 'task.log', or
+'testbot' for 'testbot.log'.
 
 For instance:
 g 0 tests.report.kernel32
@@ -531,6 +532,11 @@ sub CheckTask($$$$)
         map { $TestUnits->{wine}->{$_} = 1 } (keys %$ReportTestUnits);
       }
     }
+    elsif ($LogName =~ /^testbot\./)
+    {
+      $LogType = "testbot";
+    }
+
     if ($TaskInfo->{"$LogType.errors"})
     {
       CheckLogErrors($LogInfo, $TaskInfo->{"$LogType.errors"},
-- 
2.20.1

[tools 4/5] testbot/TestWTBS: Also check the build tasks.

[Francois Gouget]

Signed-off-by: Francois Gouget <fgouget(a)codeweavers.com>
---
 testbot/tests/TestWTBS | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/testbot/tests/TestWTBS b/testbot/tests/TestWTBS
index e3909a22d..cc3037fb7 100755
--- a/testbot/tests/TestWTBS
+++ b/testbot/tests/TestWTBS
@@ -660,7 +660,7 @@ sub CheckJobTree($)
   # which VMs to run the test on.
   if (IsMailingListJob($Job))
   {
-    foreach my $Type ("win32", "win64", "wine")
+    foreach my $Type ("build", "win32", "win64", "wine")
     {
       next if (!$HasBaseVM->{$Type});
 
-- 
2.20.1

[tools 5/5] testbot/TestWTBS: Fix the TestUnits check in case of build errors.

[Francois Gouget]

In case of a build error the TestUnits check must be skipped on Wine VMs
since the test units list is built from the test reports.

Signed-off-by: Francois Gouget <fgouget(a)codeweavers.com>
---
 testbot/tests/TestWTBS | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/testbot/tests/TestWTBS b/testbot/tests/TestWTBS
index cc3037fb7..01f46bfde 100755
--- a/testbot/tests/TestWTBS
+++ b/testbot/tests/TestWTBS
@@ -514,6 +514,12 @@ sub CheckTask($$$$)
   {
     is($Task->Status, $TaskInfo->{Status}, "Check Status of task ". TaskKeyStr($Task));
   }
+  if ($Task->Status =~ /^bad/)
+  {
+    # It makes no sense to check which test units were run in case of a build
+    # error.
+    $TestUnits->{$TaskType}->{"*skipped*"} = 1;
+  }
 
   my $ReportCount = 0;
   foreach my $LogName (@{GetLogFileNames($Task->GetDir())})
@@ -670,6 +676,8 @@ sub CheckJobTree($)
         $HasTask->{$Type} ||= 0;
         is($HasTask->{$Type}, $TypeInfo->{HasTask}, "Check the presence of $Type tasks for job ". $Job->Id);
       }
+
+      next if ($TestUnits->{$Type}->{"*skipped*"});
       if (CheckValue($TypeInfo->{TestUnits}))
       {
         foreach my $TestUnit (split / +/, $TypeInfo->{TestUnits})
-- 
2.20.1