Francois Gouget : testbot: Rework the job scheduler and add an 'off' VM state.

[Alexandre Julliard]

Module: tools
Branch: master
Commit: 2ab859b9a8dbba9a5ab8c7d295aebecfdd31fdc5
URL:    http://source.winehq.org/git/tools.git/?a=commit;h=2ab859b9a8dbba9a5ab8c7d29...

Author: Francois Gouget <fgouget(a)codeweavers.com>
Date:   Thu Oct  3 12:23:54 2013 +0200

testbot: Rework the job scheduler and add an 'off' VM state.

The old scheduler would limit the number of VMs in the running state to $MaxRunningVMs but would allow dirty base VMs to accumulate without bound. The new scheduler makes sure that no more than $MaxActiveVMs VMs are using memory, CPU or I/O resources, whether they are reverting, sleeping, running or dirty.

Furthermore the new scheduler will power off any dirty VM that it cannot revert right away to make sure it won't waste resources.
The run tasks also no longer power off the VMs when done. They now leave that task to the scheduler which will decide what to do with the VM.
The new scheduler can revert VMs while others are running. This case gets its own limit, $MaxRevertWhileRunningVMs, so the reverts do not disturb the tests.
To better exploit this it can look ahead and prepare the VMs needed for a job's next steps. And like the previous scheduler, when idle it will start a set of VMs for the next job, trying to pick the most likely to be needed: first the build VM, then base VMs, etc.
This can also gets its own limit, $MaxVMsWhenIdle, so developpers can set the TestBot to not use resources unless needed.
This can lead to having idle VMs that turn out to not be needed, for instance if the new job targets non base VMs. When that happens the scheduler can shut down some idle VMs to make room for more useful ones.

---

 testbot/bin/Engine.pl             |   12 +-
 testbot/bin/WineRunTask.pl        |    2 -
 testbot/ddl/update26.sql          |    4 +
 testbot/lib/WineTestBot/Config.pm |   17 ++-
 testbot/lib/WineTestBot/Jobs.pm   |  262 +++++++++++++++++++++++++++----------
 testbot/lib/WineTestBot/VMs.pm    |   16 ++-
 6 files changed, 221 insertions(+), 92 deletions(-)

Diff:   http://source.winehq.org/git/tools.git/?a=commitdiff;h=2ab859b9a8dbba9a5ab8c...