Hello folks, my name is Zebediah Figura, and I'll be taking over for Jeremy White today.
I'd like to start out by thanking everyone who has been working on fixing tests, with special notice to Alex Henrie who has fixed more Windows tests recently than probably anyone else combined. This mail however is not about Windows at all but Wine.
Item one: it seems to me—perhaps naïvely so, but it seems to me that we are closer than ever to having an entirely green machine (well, orange really) on the Linux end—so much so that I'm going to run through the failures that are left. Please do take the time to at least skim, since in some cases this is an RFC and what to do to fix a test isn't exactly clear even though the cause is. I also have a few other comments, so I'd appreciate your reading through to the end of this mail.
So, without further ado:
---
* kernel32:change is due to the asynchronous nature of inotify. How should this be fixed? inotify gives us one creation event and two or more notifications on the parent. It's not clear how to fix this in the test. It certainly doesn't seem possible to make the server behave like Windows.
* ole32:clipboard is explained and fixed by 140671 [1].
* quartz:filtergraph is a number of problems related to winegstreamer teardown. I have a set of patches in my local tree which should clear up most to all of these, and I'd like to start submitting these soon.
* shell32:shelllink needs some discussion, I think. The problem is that winemenubuilder is holding the file when we check for leaks. On Windows I would imagine all menu-building would be done synchronously, but on Wine this is implausible since we need to create our own copy of the link icon possibly after the link is created. At one point I suggested that menu-building should occur on prefix shutdown, which, while not really possible to do on top of the Windows API, might nevertheless be the best solution.
* urlmon:protocol is explained and fixed by 121083 [2]. This solution is less clean than is ideal, but my further attempts to synchronize more cleanly were met with other failures. I'm willing to take another look at this, although I'd appreciate help.
* user32:msg is now down to one consistent failure. This is explained and fixed by 140924/140925 [3]/[4].
* user32:win, perhaps one of the most notorious offenders, has three consistent failures left, and possibly some more intermittent ones. - Lines 3364-5 (test_SetForegroundWindow) and 9940-4 (test_LockWindowUpdate) were determined by jwhite to be the fault of the hwndMain test window being minimized during the test. I further investigated and traced this down, surprisingly enough, to a bug in fvwm, which I've reported at [5]. The bug involves a lot of steps to trigger, notably, and we can (and, I think, should) work around it by modifying the test machine configuration: either by changing the focus mode from MouseFocus, or by removing the RaiseTransient or StackTransientParent style. - Line 6838 (test_EnableWindow) is explained and fixed by 141063 [6].
* wininet:ftp, which affects both Linux and Windows, is a recent failure, as can be seen at [7]. But nothing was changed in the tests at that point. It seems more likely that something changed with respect to the ftp.winehq.org server itself, some time on or before 3 January. Could someone with knowledge of wininet or of the FTP server be implored to look into this?
Then there are the following failures. I haven't investigated any of these at all, partly because I'm as yet completely unfamiliar with the respective APIs:
* A number of Direct3D failures, all (?) of which seem to be dependent on the driver used. I'm not going to enumerate them all here, but I will make note of the fact that cw1 only needs d3d10core:device, d3d11:d3d11, and d3d9:visual, and cw2 only needs d3d9:device. All of these are consistent. * mmdevapi:capture, intermittent and rare. * mmdevapi:render, intermittent but less rare. * mshtml:htmldoc, intermittent but almost consistent. * urlmon:misc, intermittent. * xaudio2_7:xaudio2, intermittent.
While I'm certainly willing to fix every last failure myself if need be, just think how much easier it would be if people who were familiar with the code took a look at it!
Then there's one very weird set of failures: every D3D-related test from ddraw:ddraw2 onward (i.e. ddraw, dxdiagn, dxgi), on cw1-hd6800-wow32—and only that architecture—fails to print its summary line. This has been the case for since at least early November (unfortunately I lost the exact date when this started happening, but it wasn't obviously the result of any commit).
And, finally, the 64-bit failures: * d3dx9_36:math is slightly off on the spherical harmonics values due to the differences in FP math on 64-bit. Here's inviting someone who knows the math to check all of our numbers and make sure they're correct, or otherwise do as Alex Henrie did for spherical light [8]. All others are due to lack of typelib marshaling: * ieframe:ie * msxml3:domdoc * oleaut32:tmarshal * shell32:shelldispatch * user32:dialog (indirectly, since msxml3:domdoc crashes and leaves the winedbg window open)
Perhaps in these cases the best thing to do is to mark those todo_wine_if (sizeof (void *) == 8), since properly implementing the 64-bit marshaller takes more work than anyone is willing to do.
---
The ultimate goal, of course, is that we finally get a way to run testbot patches on [ideally multiple] Linux machines. Nobody needs me to tell them how important this is, but of course it's also important that we start from a clean slate, since these tests are going to be run for *every* patch, not just those that modify the tests. So while Linux is in a better state than some Windows versions at this point, it seems to me that we really do want a wholly green machine.
Along these lines it's worth noting, by the way, that except for the D3D failures, there seems to be no difference between cw1 and cw2. It seems to me it would be a good thing if we started testing multiple window managers as well (and in particular we may want to test those which see common use such as KWin and Mutter).
Also along these lines: presumably setting up the testbot to run linux tests is going to require a fair amount of work, and that work is going to be implied to be François' responsibility. I'd just like to say that I'm willing to help as much as I can, though.
Okay, so, all that said, we have another problem. I've tried running winetest on my own machine recently, in the interest of rooting out some of the rarer failures, and I've found that the reports it generates are consistently about 1.7 MB long, which is of course over the 1.5 MB limit that test.winehq.org accepts. I did some sed'ing and came up with the worst offenders as being:
test bytes % of report d3d11:d3d11 286861 16.27 ws2_32:sock 108672 6.17 kernel32:virtual 75963 4.29 gdi32:font 64700 3.61 user32:win 62114 3.52 shell32:shlexec 41235 2.34 user32:msg 37126 2.11
This isn't, I assert, something wrong with my setup. The D3D team's obviously admirable predisposition toward verbosity, and the need to print a large number of floating-point values, combined with a total of 2167 tests marked todo, has taken over almost one-sixth of the report. But even with d3d11 removed, the report just barely sneaks in under the limit, at 1475812 bytes. One might blame an excess of trace lines, as last time, but in all reality, I am inclined to think that 1.5 MB is no longer a reasonable limit for a test report. If there is an inherent limitation brought on by server storage space or processing power, well, so be it, but if not we probably will want to either raise it—or otherwise start investing energy into getting rid of existing todos.
Alright, that's finally it from me, I promise. At least for now ;-)
[1] https://source.winehq.org/patches/data/140671 [2] https://www.winehq.org/pipermail/wine-devel/2018-January/121083.html [3] https://source.winehq.org/patches/data/140924 [4] https://source.winehq.org/patches/data/140925 [5] https://github.com/fvwmorg/fvwm/issues/51 [6] https://source.winehq.org/patches/data/141063 [7] http://test.winehq.org/data/tests/wininet:ftp.html [8] https://source.winehq.org/git/wine.git/commit/07f7022fccc1c51eb4ccb4b6735e76...