May 2021 - Wine-bugs - list.winehq.org

[Bug 48651] New: Fix handling of child test processes
by WineHQ Bugzilla 03 May '21

03 May '21

https://bugs.winehq.org/show_bug.cgi?id=48651 Bug ID: 48651 Summary: Fix handling of child test processes Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- Wine's test.h provides a winetest_wait_child_process() function to wait for child processes. But when an error happens it issues some messages which the TestBot does not recognize. * If waiting for the child process fails the error message does not look like a 'Test failed' error message which it should to simplify things. * But winetest_wait_child_process() does not know on which line it was called so calling winetest_ok() would not provide the right line number. * The failure to wait for the child process should also distinguish between CreateProcess() errors (usually those are already reported by the caller), timeouts and other errors. * The Testbot does not recognize the 'child process crashed' message. This causes it to find mismatches between the number of 'Test failed' messages and the final summary. * The TestBot also does not recognize the 'failures in child process' summary line but that probably does not matter. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 4

[Bug 48671] New: Some patches may require forcing a wineprefix update
by WineHQ Bugzilla 03 May '21

03 May '21

https://bugs.winehq.org/show_bug.cgi?id=48671 Bug ID: 48671 Summary: Some patches may require forcing a wineprefix update Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- The TestBot updates the wineprefixes when it updates its base Wine source from the daily Wine commits. The rest of the time it lets Wine deal with updating the wineprefix is necessary. Furthermore Wine only updates the wineprefix when wine.inf changes, which is the case for every Wine release. The problem is that the wineprefix should also be updated when a patchset adds a WINE_REGISTRY resource to an existing dll. Failing to do so may cause the patchset tests to fail because the wineprefix is out of date. See: https://www.winehq.org/pipermail/wine-devel/2019-December/156914.html WinePrefix updates are relatively slow compared to the execution time of most tests (about 30s vs. < 1s) so it would be good not to have to do that for every task. An alternative would be to force a wineprefix update when a task has failures and gets rerun as a result (pass an option to the relevant VM-side scripts). Note: Shared Gecko and Mono install issues should be discussed on bug 48354. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 4

[Bug 48092] New: On cw-rx460 the win32 WineTest run gets interrupted before it completes
by WineHQ Bugzilla 03 May '21

03 May '21

https://bugs.winehq.org/show_bug.cgi?id=48092 Bug ID: 48092 Summary: On cw-rx460 the win32 WineTest run gets interrupted before it completes Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- cw-rx460 runs wt-daily (*), just like cw-gtx560. But when running the plain 32 bit tests on cw-rx460 (cw-gtx560-t32), wt-daily ends up submitting the test results before WineTest.exe has completed, resulting in many rejections by test.winehq.org (https://test.winehq.org/data/errors.html) WineTest appears to be interrupted most often during kernel32:loader, but sometimes it's during kernel32:debugger or ieframe:ie. And once in a while the test completes, though that's pretty rare. Maybe a test crashes the X server, causing winetest.exe to exit prematurely. wt-daily is started by cron so it does not depend on the X server and would start a second winetest command to send the (incomplete) test results. Then it would start the wow32 and wow64 tests and by that time it's possible the X server would have restarted. Interestingly wow32 and wow64 don't seem to cause the X server to crash. cw-rx460 is not a TestBot VM so this does not impact the TestBot results. But it could impact the test.winehq.org results the few times where WineTest completes (though when it does it only gets ~10 failures which is half what the other Wine machines get). So I have disabled the 32 bit WineTest runs until this is resolved. (*) https://github.com/fgouget/wt-daily -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 4

[Bug 48090] New: Add support for .com programs
by WineHQ Bugzilla 03 May '21

03 May '21

https://bugs.winehq.org/show_bug.cgi?id=48090 Bug ID: 48090 Summary: Add support for .com programs Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- The TestBot assumes that if a patch touches a directory in programs, then it must append .exe to the program's name. But Wine could also get .com programs, see for instance chcp.com: https://www.winehq.org/pipermail/wine-devel/2019-November/154371.html And in fact we already have other programs with non-standard extensions, for instance winhelp.exe16 and winoldap.mod16. So if the program's directory already has an extension the TestBot should not add one. Another option would be to use the MODULE setting in the directory's Makefile.in file. But the TestBot server does not have access to the Wine source. It would also make it harder to deal with patches adding test directories and it would be quite ugly to have to read them every time anyway. It would make it harder to deal with patches adding dlls/programs as it would require parsing the patch to retrieve the MODULE value. Another option may be to never add an extension, though then there's the risk of getting collisions between the dll and program names, and maybe it impacts other parts of the TestBot anyway. _CreateTestInfo() in testbot/lib/WineTestBot/PatchUtils.pm needs to be patched but there are impacts elsewhere. The WTBS (Wine TestBot test Suite) will also need to be updated. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 3

[Bug 48653] New: Automate checking the WTBS jobs
by WineHQ Bugzilla 03 May '21

03 May '21

https://bugs.winehq.org/show_bug.cgi?id=48653 Bug ID: 48653 Summary: Automate checking the WTBS jobs Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- The Wine TestBot test Suite (WTBS) is a set of Wine patches that can be used to check how the TestBot handles test failures, timeouts, patch series, non-Wine patches, etc: https://github.com/fgouget/wine/commits/wtbsuite The patches can either be used in isolation to test a specific aspect, or all together using formail to do a broad check for TestBot regressions. However as the test suite has become more complete the number of patches has grown and now tops 90. So a full check for regressions means checking the results of about 90 TestBot jobs for missing test failures, incorrect report validation messages, etc. This makes it pretty time consuming. So the goal is to automate at least the most basic checks. To do so the plan is to add lines of the form 'WTBS.xxx=yyy' to the commit messages of the WTBS patches. Each will define a property that should be checked in the TestBot job and tasks. Then a TestBot script can be run to go through every job, ignore the queued and running ones and those have have no associated patch. For the others read the associated patch and look for the WTBS.xxx lines. Note that for patch series, only the last set of WTBS values should be taken into account (i.e. the ones after the "Last patchset part" line). Here is a sample of the possible properties to check: * WTB.Job.Remarks The job 'title' which is derived from the patch subject line. * WTBS.Job.Status Whether to expect badpatch, completed, etc. For tasks the property names would be prefixed with the VM type: build, win (for win32 and win64), wine. This allows specifying different results for each. * WTBS.build.Status, WTBS.win.Status, WTBS.wine.Status Same as Job.Status but at the task level. * WTBS.win.Failures, WTBS.wine.Failures The number of test failures expected from the result. Note that if the Wine Test Unit is buggy we may get more test failures than expected. This is particularly an issue with Wine VMs if their mission include test=module or test=all. So it may make sense to specify WTBS.win.Failures but leave out WTBS.wine.Failures. Or maybe have a MinFailures property. Or grep the failures whose message contains 'WTBS' and only compare those to this property. * WTBS.win.NewFailures, WTBS.wine.NewFailures The number of new test failures as reported by LoadLogErrors() for each of the task's report. * WTBS.win.ReportFailures, WTBS.wine.ReportFailures In a number of cases what we're interested in is the report validation errors produced by the TestBot. Just checking the number of such errors may not be that useful. What we may want is a way to check that it reported missing failures messages, or too much data being printed, etc. This requires being able to specify a set of errors that are expected to be present for which this basic property system is not very well suited. * WTBS.win.TestUnits For each VM, collect the Step's FileName and the Task's CmdLineArg values to determine the test unit being run, and verify that this is a superset of the TestUnits list. This can be used to ensure that the TestBot ran the right set of tests (and avoids failures when a new test is added to Wine). Note that doing the same thing for Wine VMs is harder because those can have test=module in their mission(s), which changes greatly the list of tests being run. It may make sense to provide default values for some properties to not have to type too much in the commit messages. For instance for the Status properties one may expect the status to be 'completed'. But given the WTBS tests error cases a lot that may not be all that useful. Similarly it may be useful to be able to specify something like 'WTBS.all.Status' to specify the expected Status for the build, win and wine VMs all at once. But again this may be overkill. Finally the general rule should be that unspecified properties are not checked. So if 'WTBS.wine.Failures' is not specified, then the failure count of Wine VMs should not be checked. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 3

[Bug 48654] New: Automate checking the WTBS patches
by WineHQ Bugzilla 03 May '21

03 May '21

https://bugs.winehq.org/show_bug.cgi?id=48654 Bug ID: 48654 Summary: Automate checking the WTBS patches Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- See bug 48653 for a description of the WTBS and how to check the jobs created when running it on the TestBot. Some patches in the WTBS are not meant to create TestBot jobs but rather to verify that it ignores non-Wine patches, or patches that don't impact the tests or the Wine build, etc. Those still create entries in the 'wine-devel' page, that is in the TestBot's Patches table. Each entry gets the patch subject and a disposition property indicating what the TestBot did with the patch. Unfortunately non-Wine patches are not kept around so there is no way to get at the WTBS fields for those. So a TestWTBS.patches script would need to get the WTBS properties from another source. For instance it could be given the path to the WTBS mbox and get the properties from there. It would not need to understand all the intricacies of the mbox format (though with the appropriate Perl module it may be easy to parse). As a last resort, all we would care about is: * The 'From' lines to delimit emails. * The 'Subject:' lines to match the mbox emails with Patches entries. * The WTBS properties in the commit message following the Subject line. Checking the patch objects would require an extra property: * WTBS.Patch.Disposition This should be set to the expected patch disposition such as 'No patch found', 'Not a Wine patch', etc. A default disposition could be set if a 'WTBS.Job.*' or other task property is specified. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 2

[Bug 48655] New: Automate checking the WTBS emails
by WineHQ Bugzilla 03 May '21

03 May '21

https://bugs.winehq.org/show_bug.cgi?id=48655 Bug ID: 48655 Summary: Automate checking the WTBS emails Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- See bug 48653 for a description of the WTBS and how to check the jobs created when running it on the TestBot. When a 'wine-devel' job completes the TestBot sends one or two emails. * One of the emails simply sends the task reports to the patch author. * The other is only sent if a task generated new errors. The point of the TestWTBS.emails script would be to check that: 1. The TestBot sent the task report email for every Wine patch and not for any patch email. This would be based off of the WTBS.Patch.Disposition field. 2. The TestBot sent a 'found new errors' email only when new errors are expected. This would be based off of the 'WTBS.*.NewFailures' property or the expected errors list. However for both these checks the script would need access to the emails sent by the TestBot. The simplest option would be for the tester to set the WinePatchToOverride property and save the emails to a separate mbox and give the patch to that mbox to the script. As in the TestWTBS.patches case, the script would only care about a few lines: * The 'From' lines to delimit the emails. * The 'Subject' lines to match them to TestBot jobs and thus to the set of WTBS properties. Note that this would not rely on the 'TestBot job xxx results:' part since the new failures email are lacking it. Instead it would match the rest of the subject to the job's Remarks field. * The 'I think I found new failures' line to identify emails reporting new errors. The script could later be expanded to check the content of the email or attachments more in details if needed. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 2

[Bug 32216] New: WienTestBot should skip some tests on some platforms
by wine-bugs＠winehq.org 03 May '21

03 May '21

http://bugs.winehq.org/show_bug.cgi?id=32216 Bug #: 32216 Summary: WienTestBot should skip some tests on some platforms Product: Wine-Testbot Version: unspecified Platform: x86 OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: unknown AssignedTo: wine-bugs(a)winehq.org ReportedBy: fgouget(a)codeweavers.com Classification: Unclassified An example is the ieframe:webbrowser tests which it should skip on NT4 and Windows 2000 platforms. WineTest.exe is said to already have the necessary code somewhere, it's just missing from TestLauncher. See this wine-devel thread for more details: http://www.winehq.org/pipermail/wine-devel/2012-November/097819.html -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.

2 11

[Bug 50719] New: The TestBot mishandles renames
by WineHQ Bugzilla 03 May '21

03 May '21

https://bugs.winehq.org/show_bug.cgi?id=50719 Bug ID: 50719 Summary: The TestBot mishandles renames Product: Wine-Testbot Version: unspecified Hardware: x86-64 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- The TestBot analyses patches to determine whether they are Wine patches and which tests are impacted and must be rerun. But when a patch renames a test unit it still schedules a run for the renamed test unit. For instance if apphelp:apphelp is renamed to apphelp:renamed, the TestBot still schedules a run of apphelp:apphelp (in addition to the apphelp:renamed one). This means it does not keep track of Wine's current files list correctly, which also impacts identifying whether a patch is a Wine patch or not, particularly for patch series. This can be tested with the following wtbsuite tests: WTBS Check combined renaming and modifying a file (apphelp:apphelp). [2/2] WTBS S10.2 - ...before renaming it in the next part (amstream:amstream). [7/7] WTBS S5.7 - Check for rename+addition+removal conflicts (msi:suminfo). -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 2

[Bug 48208] New: Detect missing entry point errors
by WineHQ Bugzilla 03 May '21

03 May '21

https://bugs.winehq.org/show_bug.cgi?id=48208 Bug ID: 48208 Summary: Detect missing entry point errors Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs(a)winehq.org Reporter: fgouget(a)codeweavers.com Distribution: --- When a Windows test executable cannot run because of a missing test executable the TestBot remains stuck until the 2 minutes timeout. This is because Windows puts up an error dialog waiting for user input. In such a case the test report is almost empty and the issue is not obvious until one looks at the screenshot. The TestBot should detect such a situation and warn the user. One simple approach would be for the TestBot to detect that the test report only contains the TestLauncher's start and done lines and indicates a timeout. In such a case it could add a message to the .err file suggesting to check the screenshot. This would only be a heuristic though. A better approach would be for the TestLauncher to detect the error dialog. Ideally, instead of waiting for the child process for the full 2 minutes it would periodically check for the presence of this dialog, which would also speed up the task run. This approach could also be used by WineTest.exe, though presumably missing entry points would have been detected before making it into the Wine source. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.

1 8