Win16 tests - wine-devel

21 Oct 2023


      Hello!
The idea of having tests for our win16 functionality has been raised every so often, but never actualized. I would like to change that now. I think win16 has been broken several times recently, especially with large scale refactoring like the PE conversion and wow64 work, and I'd like to put tests in place to stop that happening. I'd also like to have a way to easily *write* tests for nontrivial functionality, which is currently something of an ordeal.
By my understanding, the big blocker for having win16 tests in tree is the lack of a usable win16 compiler. Existing win16 compilers are mostly hard to find; OpenWatcom is sometimes mentioned, but that only has win32 builds (as far as I'm aware), which makes it hard if not impossible to integrate into Wine.
I've spent a while over the past year or so trying a certain idea, where we write a small win16 entry point, and use generic thunks to call into win32 code, then use generic thunks the other way (WOWCallback16Ex) to call back into win16 functions. I managed to get a working PoC that actually worked on Windows XP; however, it did not work on Windows 98 [1].
In light of this, I tried a different approach. I considered but ultimately rejected the idea of introducing Win16 support into GCC [2], and instead tried an experiment to see if I could write an extremely minimalist, non-optimizing Win16 compiler myself from "scratch" [3]. It turns out that I could, in the span of what didn't end up being more than a couple months' worth of weekends.
The compiler is called Rosé, and it lives here:
https://gitlab.winehq.org/zfigura/rose
It is not fully featured; it's still missing a few notable pieces (in particular: unions, multiple code segments) but it is reasonably complete, and I was able to use it to compile a modified version of winetest.exe, and produce working executables for Windows 98.
It is therefore my proposal to use this compiler in Wine, and write in-tree Win16 tests. Here are some of the important considerations involved in this proposal:
* The compiler is small and could potentially be easily just imported into Wine. I don't have any feelings about continuing to maintain it outside of Wine—i.e. it could easily just be imported and any further development done in Wine (much like e.g. widl). "Small" here means about 15k LoC across a few files, including the builtin CRT and headers.
* NE executables, unlike both their predecessor (MZ) and successor (PE), cannot output to a controlling  console, and in general have no concept of a stdin/stdout/stderr. This does not fit well with wine tests, which are built to run with a console. There is also not much in the way of IPC available to a win16 process. They can, on the other hand, open a file and output to it.
For Marvin (or gitlab CI), this is not a huge concern, but for development it's a bit more important. In light of this we may want to write a small win32 wrapper for win16 tests, possibly one that self-extracts the win16 test.
* In order to properly test Win16 we need a contemporaneous test VM, I think ideally 98. A modern 32-bit Windows machine (of which I think we still have some) will *run* 16-bit code, and may be usable to test *some* functionality, but I do not think the results can be trusted.
This is probably a more general problem—I think our current ability to test native ddraw is similarly worthless. (ddraw is also harder, though, because we need contemporaneous *hardware* as well.) There may be other areas where our current CI is not very valuable and we really do want to test old Windows versions, although I can't think of any off the top of my head.
Thoughts and opinions? If not (as per usual) I will try to put together a patch series soon.
--Zeb
[1] In a sense this is actually very similar to how we currently implement 16-bit DLLs in Wine, except that we use a private thunk instead of the exported generic thunks [CallProcEx32W], so I spent some time refactoring various parts of krnl386 and winebuild to match my proposed design. This was going well up until I tried running the executable on Windows 98. It turns out that, unlike with NT, one cannot call any win32 functions from the 32-bit DLL without crashing. This may not be prohibitive per se [if we avoid ever making any win32 calls] but it did impose some pretty severe limitations. It's also worth mentioning that the process of refactoring was *very* convoluted, things had to be moved around very carefully in order to work.
[2] The idea of introducing compiler support into GCC has obvious advantages: not just the reuse of frontend and optimization code, but the dialect is something that will already be perfectly familiar to a Wine developer. However, it also has disadvantages: it would take a lot of time to become familiar with gcc, and I expect that segmented address spaces will be very hard to fit into the GCC code and probably not accepted upstream.
[3] In fact it is not built from scratch, but rather built from the HLSL compiler. This is largely because I am very familiar with the HLSL compiler, and familiarity is very important. I also found it far less daunting to start with the HLSL compiler, rip out everything specific to HLSL, rather than to start from scratch even using the HLSL compiler as a model.