May 2021 - Wine-Bugs - list.winehq.org

[Bug 47998] New: Better deal with random test failures
by WineHQ Bugzilla May 3, 2021

May 3, 2021

https://bugs.winehq.org/show_bug.cgi?id=47998 Bug ID: 47998 Summary: Better deal with random test failures Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- Some tests fail randomly. They should be fixed of course but the in the meantime the TestBot should try not to report them as new failures when they happen. To detect new errors the TestBot compares a task's test result with that of the latest WineTest run. So if a failure did not happen during the last WineTest run and then happens when testing a patch it will be reported as a new failure. To avoid that the TestBot should take into account not just the latest WineTest report, but all the available test reports. This way, if the random failure happened in any of those runs it will be reported as pre-existing as expected. How much of a history to take into account can then be adjusted by changing $JobPurgeDays, or adding a specific setting. Note that this means if a failure is fixed and is reintroduced soon after it will not be reported as a new failure. This scenario should be rare enough to not be an issue in practice. To implement this: 1. Store the WineTest reports in the var/latest directory with the following naming format: <vmname>-job<jobid>_<stepno>_<taskno>-<report> 2. At the start of WineRunTask and WineRunWineTest delete any reference report in the task's directory (in case the task is restarted), then make new hard links to the current set of reference reports. Handle this in LogUtils::GrabReferenceReports() so this code is shared. 3. Add LogUtils::AddReferenceReport() to deal with copying the WineTest reports to var/latest. Call this function when WineRunTask and WineRunWineTest complete. 4. In GetNewLogErrors() initially mark all errors as new. Then diff the current report with each of the reference reports located in the task's directory in turn and remove any error that's not new from the set of new errors. Note that currently the reference logs are simple copies of the original WineTest report. This means these are large files which must be parsed again to extract errors. With the current $JobPurgeDays setting there will be around 20 reference logs which will require 20 times as much log parsing. So there are two optimizations one can do, both happening when adding a new reference report (i.e. in AddReferenceReport()): a. Instead of copying the full report, save only the errors. That's all the diff needs and this should reduce the size of the files by a factor of 10 (and thus speed up parsing). b. After saving a new reference file, diff it against the old reference files. Delete any old reference file that has no error not already present in the new reference file. This will not help much if the set of failures is different with every run. But otherwise this will speed up both the parsing and diffing. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 4

[Bug 42770] New: test.winehq.org: Provide a view showing only the TestBot VMs
by wine-bugs＠winehq.org May 3, 2021

May 3, 2021

https://bugs.winehq.org/show_bug.cgi?id=42770 Bug ID: 42770 Summary: test.winehq.org: Provide a view showing only the TestBot VMs Product: WineHQ.org Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: www-unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- In Huw's words: > Something else that would be very useful is the ability to > display just the test bot VMs. These are the ones we care > about the most, and the ones that are possible to fix without > having to get a third-party to manually run a test. I find it > difficult to just keep track of these VMs when their results get > swamped by all the other machines’ failures. > > I‘d imagine this should be possible to do using static > pages. The root page (equivalent to data/index.html) > could be something like data/wtb_index.html. Clicking > on a build id would take you to data/build-id/wtb_index.html, > then clicking on an OS flavour would take you to > data/build-id/wtb_index_flavour.html. You’d also need some > way to toggle in and out of the mode. This should be pretty similar to adding the 'flat index' files mentioned in bug 39379. https://bugs.winehq.org/show_bug.cgi?id=39379 An interesting alternative would be to go the JavaScript way and make the index files, which are essentially big tables, sortable, filterable on the client-side. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

2 3

[Bug 47854] New: Block Windows 10 updates
by WineHQ Bugzilla May 3, 2021

May 3, 2021

https://bugs.winehq.org/show_bug.cgi?id=47854 Bug ID: 47854 Summary: Block Windows 10 updates Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- Windows Updates can interfere with the Wine tests in many ways: * By causing disk traffic they could slow down some tests, causing them to time out. (Some Wine tests are already somewhat slow. Obviously making them faster and/or improving QEmu's disk performance would be great but barring that preventing undue external disk traffic is necessary) * Windows installing an update at the same time WineTest is testing MSI installs could cause interference. * There is also a risk of Windows scheduling a reboot during a test, or popping up a window asking to reboot during Wine's windowing tests. And in any case Windows Updates are a waste of CPU, network and disk bandwidth since VMs will be reverted as soon as the test is completed. For these reasons pre-Windows 10 versions are configured with Windows updates disabled. But for Windows 10 that's not possible. So instead the 'Windows Update' service is disabled. However that does not survive reboots. So LibvirtTool could maybe run a command to re-disable that service when it creates a live snapshot. That command could also be repeated in WineRunTask before starting the tests. Also for Windows 10 >= 1703 it is possible to set the network connection to metered mode. This appears to prevent updates no matter what limit is set which means it's preferable to set a large limit to avoid notifications abut being past the limit. To set the network connection in metered mode: * In Windows >= 1703: Settings -> Ethernet -> Ethernet (Connected) -> 'Set as metered connection'. * In Windows >= 1809 This page also describes a regedit based approach. It's unclear if it works with older Windows 10 versions. https://www.winhelponline.com/blog/ethernet-metered-connection-windows-10-d… [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\NetworkList\DefaultMediaCost] "Ethernet"=DWORD:2 Note that this requries changing the ownership on DefaultMediaCost to the Administrators group and then back to its original owner, TrustedInstaller, aka "NT Service\TrustedInstaller" (use those Advanced buttons). -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 3

[Bug 47838] New: Add a dual-screen Windows VM
by WineHQ Bugzilla May 3, 2021

May 3, 2021

https://bugs.winehq.org/show_bug.cgi?id=47838 Bug ID: 47838 Summary: Add a dual-screen Windows VM Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Windows Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Wine has tests that deal with multi-head displays. These would need some matching multi-head test VMs. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 6

[Bug 47849] New: Simplify BuildCros() in Build.pl
by WineHQ Bugzilla May 3, 2021

May 3, 2021

https://bugs.winehq.org/show_bug.cgi?id=47849 Bug ID: 47849 Summary: Simplify BuildCros() in Build.pl Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- Build.pl is the script that's responsible for building PE executables that can be used to runt he tests on Windows VMs. Currently it's still jumping through hoops to produce PE executables (see BuildNative() and BuildCross()), which is most likely unnecessary since Wine now builds all tests as PE executables by default. So that code could likely be simplified but we would not want to lose speed advantage either: the build VM never compiles a full Wine which means patches touching headers don't cause 45+ minutes builds. Also the current code still works so it's not necessarily urgent. Side note: In fact it could be argued that the build VMs are unnecessary now since any Wine VM could produce these PE executables. There's some caveats with that though: - Wine may not produce PE executables on some platforms. For instance for a while it looked like MinGW was not usable on NetBSD, forcing the TestBot to produce only regular ELF executables. That got fixed but the issue may happen again on other Unix platforms. Maybe. - In the absence of support for load balancing it's useful to have a separate build VM to spread the load. That VM could be a repurposed Wine VM though. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 3

[Bug 47853] New: Allow testing in any locale on Wine VMs
by WineHQ Bugzilla May 3, 2021

May 3, 2021

https://bugs.winehq.org/show_bug.cgi?id=47853 Bug ID: 47853 Summary: Allow testing in any locale on Wine VMs Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- Because Wine VMs run Unix, running a test in a locale is a simple matter of setting $LANG (*), creating a new WinePrefix for good measure, and running the test normally. This does assume that the relevant locales have been configured before hand which is an easy one-time task when setting up the VM. Wine VMs already have a list of missions they perform for wine-devel patches from which we can derive a list of locales to offer on the 'Submit job' page (see bug 47852). But we would not want to run every patch through every possible locale as this would take too much time (there are dozens of locales). But extra locales could be included in the 'Submit job' page to allow debugging of these locales. (*) And $LC_ALL too??? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 3

[Bug 48651] New: Fix handling of child test processes
by WineHQ Bugzilla May 3, 2021

May 3, 2021

https://bugs.winehq.org/show_bug.cgi?id=48651 Bug ID: 48651 Summary: Fix handling of child test processes Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- Wine's test.h provides a winetest_wait_child_process() function to wait for child processes. But when an error happens it issues some messages which the TestBot does not recognize. * If waiting for the child process fails the error message does not look like a 'Test failed' error message which it should to simplify things. * But winetest_wait_child_process() does not know on which line it was called so calling winetest_ok() would not provide the right line number. * The failure to wait for the child process should also distinguish between CreateProcess() errors (usually those are already reported by the caller), timeouts and other errors. * The Testbot does not recognize the 'child process crashed' message. This causes it to find mismatches between the number of 'Test failed' messages and the final summary. * The TestBot also does not recognize the 'failures in child process' summary line but that probably does not matter. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 4

[Bug 48671] New: Some patches may require forcing a wineprefix update
by WineHQ Bugzilla May 3, 2021

May 3, 2021

https://bugs.winehq.org/show_bug.cgi?id=48671 Bug ID: 48671 Summary: Some patches may require forcing a wineprefix update Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- The TestBot updates the wineprefixes when it updates its base Wine source from the daily Wine commits. The rest of the time it lets Wine deal with updating the wineprefix is necessary. Furthermore Wine only updates the wineprefix when wine.inf changes, which is the case for every Wine release. The problem is that the wineprefix should also be updated when a patchset adds a WINE_REGISTRY resource to an existing dll. Failing to do so may cause the patchset tests to fail because the wineprefix is out of date. See: https://www.winehq.org/pipermail/wine-devel/2019-December/156914.html WinePrefix updates are relatively slow compared to the execution time of most tests (about 30s vs. < 1s) so it would be good not to have to do that for every task. An alternative would be to force a wineprefix update when a task has failures and gets rerun as a result (pass an option to the relevant VM-side scripts). Note: Shared Gecko and Mono install issues should be discussed on bug 48354. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 4

[Bug 48092] New: On cw-rx460 the win32 WineTest run gets interrupted before it completes
by WineHQ Bugzilla May 3, 2021

May 3, 2021

https://bugs.winehq.org/show_bug.cgi?id=48092 Bug ID: 48092 Summary: On cw-rx460 the win32 WineTest run gets interrupted before it completes Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- cw-rx460 runs wt-daily (*), just like cw-gtx560. But when running the plain 32 bit tests on cw-rx460 (cw-gtx560-t32), wt-daily ends up submitting the test results before WineTest.exe has completed, resulting in many rejections by test.winehq.org (https://test.winehq.org/data/errors.html) WineTest appears to be interrupted most often during kernel32:loader, but sometimes it's during kernel32:debugger or ieframe:ie. And once in a while the test completes, though that's pretty rare. Maybe a test crashes the X server, causing winetest.exe to exit prematurely. wt-daily is started by cron so it does not depend on the X server and would start a second winetest command to send the (incomplete) test results. Then it would start the wow32 and wow64 tests and by that time it's possible the X server would have restarted. Interestingly wow32 and wow64 don't seem to cause the X server to crash. cw-rx460 is not a TestBot VM so this does not impact the TestBot results. But it could impact the test.winehq.org results the few times where WineTest completes (though when it does it only gets ~10 failures which is half what the other Wine machines get). So I have disabled the 32 bit WineTest runs until this is resolved. (*) https://github.com/fgouget/wt-daily -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 4

[Bug 48090] New: Add support for .com programs
by WineHQ Bugzilla May 3, 2021

May 3, 2021

https://bugs.winehq.org/show_bug.cgi?id=48090 Bug ID: 48090 Summary: Add support for .com programs Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- The TestBot assumes that if a patch touches a directory in programs, then it must append .exe to the program's name. But Wine could also get .com programs, see for instance chcp.com: https://www.winehq.org/pipermail/wine-devel/2019-November/154371.html And in fact we already have other programs with non-standard extensions, for instance winhelp.exe16 and winoldap.mod16. So if the program's directory already has an extension the TestBot should not add one. Another option would be to use the MODULE setting in the directory's Makefile.in file. But the TestBot server does not have access to the Wine source. It would also make it harder to deal with patches adding test directories and it would be quite ugly to have to read them every time anyway. It would make it harder to deal with patches adding dlls/programs as it would require parsing the patch to retrieve the MODULE value. Another option may be to never add an extension, though then there's the risk of getting collisions between the dll and program names, and maybe it impacts other parts of the TestBot anyway. _CreateTestInfo() in testbot/lib/WineTestBot/PatchUtils.pm needs to be patched but there are impacts elsewhere. The WTBS (Wine TestBot test Suite) will also need to be updated. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 3