[Bug 39487] New: Make step dependencies more flexible

https://bugs.winehq.org/show_bug.cgi?id=39487

Bug ID: 39487
           Summary: Make step dependencies more flexible
           Product: Wine-Testbot
           Version: unspecified
          Hardware: x86
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

The TestBot jobs are split into steps and tasks.
 * The steps are numbered and must be executed in order. Each step has an
associated file.
 * Each step has one or more tasks which can all be run in parallel.

Typically the first step is the build step associated to the patch. It produces
the 32 and 64 bit executables for the following steps hence why it has to be
executed first. When it fails the other steps are skipped which is not the case
for the other steps.

So the general structure is:
 Step 1 - patch.diff - Build
 Step 2 - test_32.exe - One task per VM
 Step 3 - test_64.exe - One task per VM

So one source of inefficiency is that the 64 bit tests cannot start until all
the 32 bit tests have completed. It does not necessarily make the overall job
execution longer but it can leave a VM host idle while it waits for the
remaining 32 bit tests to complete. And if a developer is watching the job's
page he will not know of 64 bit failures until then.

So while the build step has to come before all others, that condition should be
relaxed for the other steps.

The database schema presented in bug 39412 adds a DependencyNo field to the
Step table so that step dependencies can be made explicit. This would allow
making both test steps depend on build one. Note that with this scheme a given
step could not directly depend on multiple other steps but that does not seem
needed.

It would also make more explicit the fact that test steps cannot be run if the
build one fails instead of hard-coding that behavior in Jobs::UpdateStatus().

[Bug 39487] Make step dependencies more flexible

https://bugs.winehq.org/show_bug.cgi?id=39487

François Gouget  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #1 from François Gouget  ---
This is done:

commit 4080be2cabb39f70019785ed4e49db68d6b5b310
Author: Francois Gouget 
Date:   Mon Feb 26 01:40:25 2018 +0100

testbot: Allow explicitly specifying a Step's dependency.

This allows the TestBot to start running the 64 bit tests even if there
    are still some 32 bit tests to run.
    It also lets the TestBot more freely schedule the tasks when a job
    involves many test units (and thus many steps).

Signed-off-by: Francois Gouget 
    Signed-off-by: Alexandre Julliard

Since this patch there has been some slightly broken scheduling on vm4 where
the build VM is. The symptom is that instead of remaining idle the build VM
goes through a string of revert -> sleeping -> idle -> sacrifice state changes
while the tests are being run on the other hosts.

Fortunately this does not break anything. It just means vm4 performs needless
work (and distorts that host's business statistics). Also I think this is not a
bug in the above patch but a preexisting bug in the database access framework
that just got exposed by this patch.

[Bug 39487] Make step dependencies more flexible

https://bugs.winehq.org/show_bug.cgi?id=39487

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #2 from Austin English  ---
Closing.