October 2023 - Wine-Devel - list.winehq.org

Re: A case for policy changes to get to zero test failures

by Alexandre Julliard

Francois Gouget <fgouget(a)codeweavers.com> writes: > * The other policy changes are still to be determined. Thank you for raising the issue. I understand your frustration that after all these years we still can't achieve zero failures, but I'm not envisioning drastic policy changes at this point, particularly not extra constraints on developers. We all know that fixing tests can be hard even for the best developers, so punishing people for not fixing tests, by reverting their changes or blocking their MRs, would be counterproductive IMHO. There's a balance to strike between the efforts needed to fix the tests, and the efforts needed to develop new features and fix bugs in real apps. A successful test suite is important, but it's not important enough to sacrifice progress in other areas. We also have to look at the positive side: we have millions of tests that are being executed and succeed on every commit, and they are catching many potential regressions. While the failures are annoying, they are only a tiny percentage of the tests, and they don't make the whole test suite useless. That's also why I don't want to disable an entire test file when only a couple of tests fail. However, it's OK to disable specific tests, using flaky() or todo(), if they are too hard to fix properly. So I think we have to accept that test failures and regressions are the price to pay for making progress. We all need to do our best to minimize them and keep them under control, but given the complexity of what we are doing with Wine, we can't always achieve perfection. I'm also hoping that, as the pool of paid developers grows, we will have more room to allocate some paid developer time to fixing the tests. -- Alexandre Julliard julliard(a)winehq.org

Oct. 31, 2023

Re: A case for policy changes to get to zero test failures

by Giovanni Mascellani

Hi, Il 23/10/23 14:29, Francois Gouget ha scritto: > * Replicating failures can sometimes be frustratingly hard. Even if you > happen to have access to the CI environment! [2] > > * And usually one does not have access to the CI environment. That's the > the case for the Windows environments for licensing reasons. On the > Linux side the GitLab CI should mean progress but replicating it needs > to be made much simpler before that's effective. In the meantime that > means no debugger, no watching what happens on screen while the test > runs, little poking into the test environment entrails (short of > writing a dedicated program to do the poking), etc. In short it's > remote debugging. This is indeed a pain point. So much that at some point I even began to write a tentative solution to that, which so far I have only shared inside CodeWeavers. There is in fact (as far as I am aware) no reason why it should not be made available publicly, so here it is, I just released it: https://gitlab.winehq.org/giomasce/minitestbot Here is the train of thoughts that brought to MiniTestBot: * As you say, accessing the TestBot/CI environment is hard: the interface is limited and people have to share the available time, so if many people are hitting the TestBot/CI at the same time they have to wait for each other's job, which can be frustrating. * Much better it would be if you could run the TestBot/CI's environment directly on your hardware. Then you don't compete with others anymore and you can interact with your VM however you please, including changing settings and installing stuff. * As you say, the problem is that we can't just distribute Windows images. But we can distribute the scripts used to generate them, which is essentially what MiniTestBot does! The MiniTestBot user needs to independently provide the Windows ISO (Microsoft distributes them for free, at least for some Windows versions) and is directly responsible for complying with licenses, while distributing the scripts does not infringe any copyright, as they are all free software and free of Microsoft intellectual property. Though I am not a lawyer, so don't take this as legal advice! :-) * If the TestBot/CI used images generated with these scripts (or, rather, what they would become after having made better) we'd have both the possibility to run tests in the cloud, which is handy for many reasons, and the possibility to run them on premises in the same environment, which is handy in other cases. I hope that would help to remove at least some of the frustration of debugging tests on Windows. In practice MiniTestBot is a few scripts that take a Windows image and (nearly) automatically create an image out of it, together with other scripts to submit job to be automatically be executed inside that image, TestBot style. It is really a proof of concept at this stage, but I'm already finding it useful for development. At least another person in CodeWeavers told me they find it useful. It would require some more work to be integrated in the TestBot/CI, but that work shouldn't be impossible to do. What do you think of this approach? Also, I am happy to receive comments, suggestions and MRs (yes, those too!) for MiniTestBot. Gio.

Oct. 30, 2023

Re: A case for policy changes to get to zero test failures

by Francois Gouget

On Mon, 23 Oct 2023, Francois Gouget wrote: [...] > So the conclusion I have come to is that making further and lasting > progress will require policy changes to incentivize work on the tests. * The goal is to get rid of the TestBot so the bridge to GitLab will not be restored in any form. There will also be no further TestBot development. * Wine developers are on their own to figure out if their commits cause new failures, and if they do they are on their own to figure out how to reproduce the issues. * The other policy changes are still to be determined. -- Francois Gouget <fgouget(a)codeweavers.com>

Oct. 30, 2023

Re: The future of WineConf

by Huw Davies

On 30 Oct 2023, at 10:50, Alexandre Julliard <julliard(a)winehq.org> wrote: > A number of Wine developers will be there in February. We'll walk around > the conference, but since finding people at FOSDEM proper is not exactly > easy, the plan is to organize an event at a local bar on Saturday > evening. Anybody who'd like to hang out will be more than welcome to > join us there. I've booked a table at the BrewDog [1] for 18:30 on Sat 3rd Feb. Looking forward to seeing folks there! Huw [1] https://www.brewdog.com/usa/brewdog-brussels

Oct. 30, 2023

Re: The future of WineConf

by Alexandre Julliard

Henri Verbeet <hverbeet(a)gmail.com> writes: > So practically speaking, should I be planning to travel to Brussels to > meet Wine developers this February? Yes, for now the plan is to use FOSDEM as a substitute for the social part of WineConf. A number of Wine developers will be there in February. We'll walk around the conference, but since finding people at FOSDEM proper is not exactly easy, the plan is to organize an event at a local bar on Saturday evening. Anybody who'd like to hang out will be more than welcome to join us there. -- Alexandre Julliard julliard(a)winehq.org

Oct. 30, 2023

Wine staging 8.19 release

by Alistair Leslie-Hughes

Binary packages for various distributions will be available from: https://www.winehq.org/download Summary since last release * Rebased to current wine 8.18 (493 patches are applied to wine vanilla) Upstreamed (Either directly from staging or fixed with a similar patch). * ntdll: Don't use Wine frames during exception processing on x64. Added: * None Updated: * vkd3d-latest to latest. * fltmgr.sys-FltBuildDefaultSecurityDescriptor Where can you help * Run Steam/Battle.net/GOG/UPlay/Epic * Test your favorite game. * Test your favorite applications. * Improve staging patches and get them accepted upstream. * Suggest patches to be included in staging. As always, if you find a bug, please report it via https://bugs.winehq.org Best Regards Alistair.

Oct. 30, 2023

Re: Win16 tests

by Fabian Maurer

Hello Zeb, I'd love to see Win16 tests in upstream Wine, so thanks for working on this! This would make picking changes from winevdm (the fork) into Wine a lot easier I think. > OpenWatcom is sometimes mentioned, but that only has win32 builds > (as far as I'm aware), which makes it hard if not impossible to integrate > into Wine. You can compile OpenWatcom on Linux, or did I misunderstand that? I always thought the problem with OpenWatcom was that it's not as easily available as gcc and to avoid the dependency. > The compiler is called Rosé, and it lives here: > > https://gitlab.winehq.org/zfigura/rose > > It is not fully featured; it's still missing a few notable pieces (in > particular: unions, multiple code segments) but it is reasonably complete, > and I was able to use it to compile a modified version of winetest.exe, and > produce working executables for Windows 98. Could you provide a minimal example of a working program that uses MessageBoxA or printf? > * The compiler is small and could potentially be easily just imported into > Wine. I don't have any feelings about continuing to maintain it outside of > Wine—i.e. it could easily just be imported and any further development done > in Wine (much like e.g. widl). "Small" here means about 15k LoC across a > few files, including the builtin CRT and headers. Not that my opinion mattered a lot, but I would like having it under the Wine umbrella, makes my setup easier. > * NE executables, unlike both their predecessor (MZ) and successor (PE), > cannot output to a controlling console, and in general have no concept of > a stdin/stdout/stderr. This does not fit well with wine tests, which are > built to run with a console. There is also not much in the way of IPC > available to a win16 process. They can, on the other hand, open a file and > output to it. Maybe a stupid question, but won't dos console interrupts works for console? > * In order to properly test Win16 we need a contemporaneous test VM, I think > ideally 98. A modern 32-bit Windows machine (of which I think we still have > some) will *run* 16-bit code, and may be usable to test *some* > functionality, but I do not think the results can be trusted. Hm, I always thought XP would be a good win16 test-target. But then again, there is no XP VM either. > [2] The idea of introducing compiler support into GCC has obvious > advantages: not just the reuse of frontend and optimization code, but the > dialect is something that will already be perfectly familiar to a Wine > developer. However, it also has disadvantages: it would take a lot of time > to become familiar with gcc, and I expect that segmented address spaces > will be very hard to fit into the GCC code and probably not accepted > upstream. Since gcc is capable of generating "fake" 16bit code using -m16, is this something that could be reused? Although I don't think one could get segmentation to work without way too much effort. Regards, Fabian Maurer

Oct. 23, 2023

Win16 tests

by Zeb Figura

Hello! The idea of having tests for our win16 functionality has been raised every so often, but never actualized. I would like to change that now. I think win16 has been broken several times recently, especially with large scale refactoring like the PE conversion and wow64 work, and I'd like to put tests in place to stop that happening. I'd also like to have a way to easily *write* tests for nontrivial functionality, which is currently something of an ordeal. By my understanding, the big blocker for having win16 tests in tree is the lack of a usable win16 compiler. Existing win16 compilers are mostly hard to find; OpenWatcom is sometimes mentioned, but that only has win32 builds (as far as I'm aware), which makes it hard if not impossible to integrate into Wine. I've spent a while over the past year or so trying a certain idea, where we write a small win16 entry point, and use generic thunks to call into win32 code, then use generic thunks the other way (WOWCallback16Ex) to call back into win16 functions. I managed to get a working PoC that actually worked on Windows XP; however, it did not work on Windows 98 [1]. In light of this, I tried a different approach. I considered but ultimately rejected the idea of introducing Win16 support into GCC [2], and instead tried an experiment to see if I could write an extremely minimalist, non-optimizing Win16 compiler myself from "scratch" [3]. It turns out that I could, in the span of what didn't end up being more than a couple months' worth of weekends. The compiler is called Rosé, and it lives here: https://gitlab.winehq.org/zfigura/rose It is not fully featured; it's still missing a few notable pieces (in particular: unions, multiple code segments) but it is reasonably complete, and I was able to use it to compile a modified version of winetest.exe, and produce working executables for Windows 98. It is therefore my proposal to use this compiler in Wine, and write in-tree Win16 tests. Here are some of the important considerations involved in this proposal: * The compiler is small and could potentially be easily just imported into Wine. I don't have any feelings about continuing to maintain it outside of Wine—i.e. it could easily just be imported and any further development done in Wine (much like e.g. widl). "Small" here means about 15k LoC across a few files, including the builtin CRT and headers. * NE executables, unlike both their predecessor (MZ) and successor (PE), cannot output to a controlling console, and in general have no concept of a stdin/stdout/stderr. This does not fit well with wine tests, which are built to run with a console. There is also not much in the way of IPC available to a win16 process. They can, on the other hand, open a file and output to it. For Marvin (or gitlab CI), this is not a huge concern, but for development it's a bit more important. In light of this we may want to write a small win32 wrapper for win16 tests, possibly one that self-extracts the win16 test. * In order to properly test Win16 we need a contemporaneous test VM, I think ideally 98. A modern 32-bit Windows machine (of which I think we still have some) will *run* 16-bit code, and may be usable to test *some* functionality, but I do not think the results can be trusted. This is probably a more general problem—I think our current ability to test native ddraw is similarly worthless. (ddraw is also harder, though, because we need contemporaneous *hardware* as well.) There may be other areas where our current CI is not very valuable and we really do want to test old Windows versions, although I can't think of any off the top of my head. Thoughts and opinions? If not (as per usual) I will try to put together a patch series soon. --Zeb [1] In a sense this is actually very similar to how we currently implement 16-bit DLLs in Wine, except that we use a private thunk instead of the exported generic thunks [CallProcEx32W], so I spent some time refactoring various parts of krnl386 and winebuild to match my proposed design. This was going well up until I tried running the executable on Windows 98. It turns out that, unlike with NT, one cannot call any win32 functions from the 32-bit DLL without crashing. This may not be prohibitive per se [if we avoid ever making any win32 calls] but it did impose some pretty severe limitations. It's also worth mentioning that the process of refactoring was *very* convoluted, things had to be moved around very carefully in order to work. [2] The idea of introducing compiler support into GCC has obvious advantages: not just the reuse of frontend and optimization code, but the dialect is something that will already be perfectly familiar to a Wine developer. However, it also has disadvantages: it would take a lot of time to become familiar with gcc, and I expect that segmented address spaces will be very hard to fit into the GCC code and probably not accepted upstream. [3] In fact it is not built from scratch, but rather built from the HLSL compiler. This is largely because I am very familiar with the HLSL compiler, and familiarity is very important. I also found it far less daunting to start with the HLSL compiler, rip out everything specific to HLSL, rather than to start from scratch even using the HLSL compiler as a model.

Oct. 21, 2023

Re: GitLab CI's Most Wanted list

by Zeb Figura

On Friday, 20 October 2023 03:49:50 CDT Francois Gouget wrote: > > Processing the failures-mr.txt file for the past month (see previous > email) points to the following test units as plagging the most merge > requests: > > 100 msxml3:domdoc (55720) > 87 amstream:amstream (55725) > 81 quartz:filtergraph (55758) > 44 build:mac > 26 d2d1:d2d1 (55765) > 19 winmm:mci (55605) > 15 user32:msg (55786, mostly) > 13 user32:win (55766) > 12 psapi:psapi_main (55744, fixed on day 3) > 11 wscript.exe:run (55746, fixed on day 3) > 11 tasklist.exe:tasklist (55745, fixed on day 3) > > Good hunting! > > I'll look into the DirectShow ones (quartz, amstream, mci).

Oct. 20, 2023

Re: GitLab CI's Most Wanted list

by Francois Gouget

On Fri, 20 Oct 2023, Eric Pouech wrote: [...] > I've tried to repro a couple of them locally, recreating the docker image with > the docker script... to no avail I have a question on that subject: how do you run the Wine tests in your local docker image? The reason I ask is that reproducing GitLab CI's container and using it for builds is pretty easy. But running the tests in it requires starting X and the commands (in test.yml) for that are not directly usable outside the GitLab environment. So do you copy/paste all the xinitrc, startx, wine-mono & co stuff to a script, append the ./wine command you want to run at the end and do something like "docker run gitlab-debian12 ./myscript"? That also assumes mounting the current directory somewhere in the container but having a script that does echo >~/.xinitrc sounds like a disaster waiting to happen if it's accidentally run in the wrong environment. Or did you add a RUN directive in the Dockerfile to start X and then 'docker exec' your test in it? I feel like it would be nice if tools/gitlab contained a script or something to make this whole process easier. Developers have actually asked me [1] "how can I replicate the GitLab CI environment" a few times already. [1] Why ask me? Am I the GitLab CI maintainer now? -- Francois Gouget <fgouget(a)codeweavers.com>

Oct. 20, 2023