https://bugs.winehq.org/show_bug.cgi?id=48191
Bug ID: 48191
Summary: Add a captcha to the feedback page
Product: Wine-Testbot
Version: unspecified
Hardware: x86
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: unknown
Assignee: wine-bugs(a)winehq.org
Reporter: fgouget(a)codeweavers.com
Distribution: ---
We are now regularly getting spam on the captcha page. Spammers really have
time to waste!
So we should add a captcha. Despite its simplicity the one on the registration
page still seems to work so it should be a simple matter of porting it over.
--
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=48035
Bug ID: 48035
Summary: Speed up the new failures detection
Product: Wine-Testbot
Version: unspecified
Hardware: x86
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: unknown
Assignee: wine-bugs(a)winehq.org
Reporter: fgouget(a)codeweavers.com
Distribution: ---
When a task completes the TestBot distinguishes new failures from preexisting
ones by comparing the task's report to the latest WineTest report. Any failure
present in the task's report which is not present in the WineTest report is
new.
This means parsing both reports to extract the errors and diffing them. This
needs to be done whenever we need the list of new failures which is:
* When sending a notification to the developer when a wine-patches job
completes.
* On the JobDetails page.
But the WineTest reports are pretty big (~600KB on Windows, ~1.5MB on Wine)
which translates into non negligible processing time. For instance on my
development environment loading the JobDetails page for a job with 1 Windows
report and 8 Wine reports takes about 0.2 s without the reference WineTest
reports, but close to 2 s when they are present.
So it would help to only store the WineTest failures instead of the full report
(optimisation a in bug 47998). This would cut down not only on the amount of
data to read, but also on the required parsing. Testing this in the same
development environment indicates this could bring the JobDetails processing
time down to about 0.3 s.
--
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=47998
Bug ID: 47998
Summary: Better deal with random test failures
Product: Wine-Testbot
Version: unspecified
Hardware: x86
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: unknown
Assignee: wine-bugs(a)winehq.org
Reporter: fgouget(a)codeweavers.com
Distribution: ---
Some tests fail randomly. They should be fixed of course but the in the
meantime the TestBot should try not to report them as new failures when they
happen.
To detect new errors the TestBot compares a task's test result with that of the
latest WineTest run. So if a failure did not happen during the last WineTest
run and then happens when testing a patch it will be reported as a new failure.
To avoid that the TestBot should take into account not just the latest WineTest
report, but all the available test reports. This way, if the random failure
happened in any of those runs it will be reported as pre-existing as expected.
How much of a history to take into account can then be adjusted by changing
$JobPurgeDays, or adding a specific setting.
Note that this means if a failure is fixed and is reintroduced soon after it
will not be reported as a new failure. This scenario should be rare enough to
not be an issue in practice.
To implement this:
1. Store the WineTest reports in the var/latest directory with the following
naming format:
<vmname>-job<jobid>_<stepno>_<taskno>-<report>
2. At the start of WineRunTask and WineRunWineTest delete any reference report
in the task's directory (in case the task is restarted), then make new hard
links to the current set of reference reports. Handle this in
LogUtils::GrabReferenceReports() so this code is shared.
3. Add LogUtils::AddReferenceReport() to deal with copying the WineTest reports
to var/latest. Call this function when WineRunTask and WineRunWineTest
complete.
4. In GetNewLogErrors() initially mark all errors as new. Then diff the current
report with each of the reference reports located in the task's directory in
turn and remove any error that's not new from the set of new errors.
Note that currently the reference logs are simple copies of the original
WineTest report. This means these are large files which must be parsed again to
extract errors. With the current $JobPurgeDays setting there will be around 20
reference logs which will require 20 times as much log parsing. So there are
two optimizations one can do, both happening when adding a new reference report
(i.e. in AddReferenceReport()):
a. Instead of copying the full report, save only the errors. That's all the
diff needs and this should reduce the size of the files by a factor of 10 (and
thus speed up parsing).
b. After saving a new reference file, diff it against the old reference files.
Delete any old reference file that has no error not already present in the new
reference file. This will not help much if the set of failures is different
with every run. But otherwise this will speed up both the parsing and diffing.
--
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=42770
Bug ID: 42770
Summary: test.winehq.org: Provide a view showing only the
TestBot VMs
Product: WineHQ.org
Version: unspecified
Hardware: x86
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: www-unknown
Assignee: wine-bugs(a)winehq.org
Reporter: fgouget(a)codeweavers.com
Distribution: ---
In Huw's words:
> Something else that would be very useful is the ability to
> display just the test bot VMs. These are the ones we care
> about the most, and the ones that are possible to fix without
> having to get a third-party to manually run a test. I find it
> difficult to just keep track of these VMs when their results get
> swamped by all the other machines’ failures.
>
> I‘d imagine this should be possible to do using static
> pages. The root page (equivalent to data/index.html)
> could be something like data/wtb_index.html. Clicking
> on a build id would take you to data/build-id/wtb_index.html,
> then clicking on an OS flavour would take you to
> data/build-id/wtb_index_flavour.html. You’d also need some
> way to toggle in and out of the mode.
This should be pretty similar to adding the 'flat index' files mentioned in bug
39379.
https://bugs.winehq.org/show_bug.cgi?id=39379
An interesting alternative would be to go the JavaScript way and make the index
files, which are essentially big tables, sortable, filterable on the
client-side.
--
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=47854
Bug ID: 47854
Summary: Block Windows 10 updates
Product: Wine-Testbot
Version: unspecified
Hardware: x86
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: unknown
Assignee: wine-bugs(a)winehq.org
Reporter: fgouget(a)codeweavers.com
Distribution: ---
Windows Updates can interfere with the Wine tests in many ways:
* By causing disk traffic they could slow down some tests, causing them to time
out. (Some Wine tests are already somewhat slow. Obviously making them faster
and/or improving QEmu's disk performance would be great but barring that
preventing undue external disk traffic is necessary)
* Windows installing an update at the same time WineTest is testing MSI
installs could cause interference.
* There is also a risk of Windows scheduling a reboot during a test, or popping
up a window asking to reboot during Wine's windowing tests.
And in any case Windows Updates are a waste of CPU, network and disk bandwidth
since VMs will be reverted as soon as the test is completed.
For these reasons pre-Windows 10 versions are configured with Windows updates
disabled. But for Windows 10 that's not possible.
So instead the 'Windows Update' service is disabled. However that does not
survive reboots. So LibvirtTool could maybe run a command to re-disable that
service when it creates a live snapshot. That command could also be repeated in
WineRunTask before starting the tests.
Also for Windows 10 >= 1703 it is possible to set the network connection to
metered mode. This appears to prevent updates no matter what limit is set which
means it's preferable to set a large limit to avoid notifications abut being
past the limit.
To set the network connection in metered mode:
* In Windows >= 1703: Settings -> Ethernet -> Ethernet (Connected) -> 'Set as
metered connection'.
* In Windows >= 1809
This page also describes a regedit based approach. It's unclear if it works
with older Windows 10 versions.
https://www.winhelponline.com/blog/ethernet-metered-connection-windows-10-d…
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows
NT\CurrentVersion\NetworkList\DefaultMediaCost]
"Ethernet"=DWORD:2
Note that this requries changing the ownership on DefaultMediaCost to the
Administrators group and then back to its original owner, TrustedInstaller, aka
"NT Service\TrustedInstaller" (use those Advanced buttons).
--
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=47838
Bug ID: 47838
Summary: Add a dual-screen Windows VM
Product: Wine-Testbot
Version: unspecified
Hardware: x86
OS: Windows
Status: NEW
Severity: normal
Priority: P2
Component: unknown
Assignee: wine-bugs(a)winehq.org
Reporter: fgouget(a)codeweavers.com
Wine has tests that deal with multi-head displays. These would need some
matching multi-head test VMs.
--
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=47849
Bug ID: 47849
Summary: Simplify BuildCros() in Build.pl
Product: Wine-Testbot
Version: unspecified
Hardware: x86
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: unknown
Assignee: wine-bugs(a)winehq.org
Reporter: fgouget(a)codeweavers.com
Distribution: ---
Build.pl is the script that's responsible for building PE executables that can
be used to runt he tests on Windows VMs.
Currently it's still jumping through hoops to produce PE executables (see
BuildNative() and BuildCross()), which is most likely unnecessary since Wine
now builds all tests as PE executables by default.
So that code could likely be simplified but we would not want to lose speed
advantage either: the build VM never compiles a full Wine which means patches
touching headers don't cause 45+ minutes builds. Also the current code still
works so it's not necessarily urgent.
Side note:
In fact it could be argued that the build VMs are unnecessary now since any
Wine VM could produce these PE executables. There's some caveats with that
though:
- Wine may not produce PE executables on some platforms. For instance for a
while it looked like MinGW was not usable on NetBSD, forcing the TestBot to
produce only regular ELF executables. That got fixed but the issue may happen
again on other Unix platforms. Maybe.
- In the absence of support for load balancing it's useful to have a separate
build VM to spread the load. That VM could be a repurposed Wine VM though.
--
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=47853
Bug ID: 47853
Summary: Allow testing in any locale on Wine VMs
Product: Wine-Testbot
Version: unspecified
Hardware: x86
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: unknown
Assignee: wine-bugs(a)winehq.org
Reporter: fgouget(a)codeweavers.com
Distribution: ---
Because Wine VMs run Unix, running a test in a locale is a simple matter of
setting $LANG (*), creating a new WinePrefix for good measure, and running the
test normally.
This does assume that the relevant locales have been configured before hand
which is an easy one-time task when setting up the VM.
Wine VMs already have a list of missions they perform for wine-devel patches
from which we can derive a list of locales to offer on the 'Submit job' page
(see bug 47852). But we would not want to run every patch through every
possible locale as this would take too much time (there are dozens of locales).
But extra locales could be included in the 'Submit job' page to allow debugging
of these locales.
(*) And $LC_ALL too???
--
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=48651
Bug ID: 48651
Summary: Fix handling of child test processes
Product: Wine-Testbot
Version: unspecified
Hardware: x86
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: unknown
Assignee: wine-bugs(a)winehq.org
Reporter: fgouget(a)codeweavers.com
Distribution: ---
Wine's test.h provides a winetest_wait_child_process() function to wait for
child processes. But when an error happens it issues some messages which the
TestBot does not recognize.
* If waiting for the child process fails the error message does not look like a
'Test failed' error message which it should to simplify things.
* But winetest_wait_child_process() does not know on which line it was called
so calling winetest_ok() would not provide the right line number.
* The failure to wait for the child process should also distinguish between
CreateProcess() errors (usually those are already reported by the caller),
timeouts and other errors.
* The Testbot does not recognize the 'child process crashed' message. This
causes it to find mismatches between the number of 'Test failed' messages and
the final summary.
* The TestBot also does not recognize the 'failures in child process' summary
line but that probably does not matter.
--
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=48671
Bug ID: 48671
Summary: Some patches may require forcing a wineprefix update
Product: Wine-Testbot
Version: unspecified
Hardware: x86
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: unknown
Assignee: wine-bugs(a)winehq.org
Reporter: fgouget(a)codeweavers.com
Distribution: ---
The TestBot updates the wineprefixes when it updates its base Wine source from
the daily Wine commits. The rest of the time it lets Wine deal with updating
the wineprefix is necessary. Furthermore Wine only updates the wineprefix when
wine.inf changes, which is the case for every Wine release.
The problem is that the wineprefix should also be updated when a patchset adds
a WINE_REGISTRY resource to an existing dll. Failing to do so may cause the
patchset tests to fail because the wineprefix is out of date.
See:
https://www.winehq.org/pipermail/wine-devel/2019-December/156914.html
WinePrefix updates are relatively slow compared to the execution time of most
tests (about 30s vs. < 1s) so it would be good not to have to do that for every
task. An alternative would be to force a wineprefix update when a task has
failures and gets rerun as a result (pass an option to the relevant VM-side
scripts).
Note: Shared Gecko and Mono install issues should be discussed on bug 48354.
--
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.