Hi Zebediah,
I really think we should at least experiment and evaluate using winelib first. My previous mail didn't really explain details behind this and was a bit messy, so let me try now.
First, the name mingw or mingw-w64 is used to name different things depending on the context. For sake of this mail, let me use it to call the complete toolchain for (cross, but not only) compiling a Window target. mingw based on GCC contains following parts from different projects (LLVM mingw variant is similar, except that llvm/clang is always a cross compiler, so this part doesn't need to be built with mingw in mind and instead a fairly regular build can be used):
- GCC + binutils tools built with mingw as a target (from gcc and binutils tree) - libgcc cross compiled for mingw target (from gcc tree, but depends on crt part of mingw-w64) - crt headers (from mingw-w64 repo) - crt static libraries and startup files (libmingwex, libmingw32, libmsvcrt, etc, from mingw-w64 repo) - platform headers (from mingw-w64 repo) - platform importlibs (from mingw-w64 repo) - optionally winpthreads, which messes with crt headers and crt libs (from mingw-w64 repo) - libstdc++ and a few more component not important for sake of this conversation
Wine is not a typical mingw user, because it actually implements the platform for which it builds. As such, we need to provide majority of the stuff that other projects would use from mingw-w64. Initially, Wine used an exotic mix for crt: it used its own headers and msvcrt importlib, but mingw-w64 for the rest of static libs and startup files. It meant that Wine dependent on mingw-w64 internals that it shouldn't touch. mingw-w64 assumes that crt headers + crt libs are shipped together and it has the right to do so. It worked, but it was fragile. Anything that Wine didn't take into account (like configuring mingw-w64 to use ucrt by default) could cause a problem. It's probably just a matter of time when wine-5.0 will not build with future mingw-w64 releases and it will not be mingw-w64's fault.
We looked for a solution to that problem. One way to solve this would be tightening projects relationships. It's wasn't clear how technically that could look, but we had to put that idea aside for non-technical reasons anyway.
The other solution is to not mix crt parts at all. This required Wine to provide the few missing bits (mostly startup files) by itself. I'd say that it makes sense: we provide similar things for ELF builds anyway and we provide msvcrt implementation and headers, it feels natural to provide static library bits as well. Since February (or August last year for majority of the codebase), Wine does not use any part of mingw-w64 repo directly at all. We always use our own headers and -nodefaultlibs. Wine cross toolchain dependency is reduced to just GCC + binutils + libgcc (and mingw-w64 as build time libgcc dependency) from the list above. That's what you get with winegcc -mno-cygwin (without -mno-cygwin, we still use the exotic mix for mingw targets and we still use that for PE-only builds).
So now that we have this powerful tool and successfully use it for Wine build, it feels natural and technically right to use it for the rest of Wine ecosystem. What you propose would instead bring back mixing crt parts. Linking current Wine against external mingw-compiled (as opposed to winegcc -mno-cygwin) library is essentially linking a static lib with a different ABI. Headers should mostly work, but it's still shaky.
I didn't try winegcc yet with non-Wine code myself mostly because I still think there are things that we should get right first. My ucrt work was partially motivated by this. We should be able to default to ucrt in winegcc now (we already do that for Wine in makedep, so I'd say we're ready for that). There are a few other things, but we're getting there.
On 13.04.2020 01:18, Zebediah Figura wrote:
So, I figure it's probably worth quantifying what we'd gain this way (as well as how many libraries we'd potentially have to build or maintain). As I see it, the libraries that can be built as PE (unless I misunderstand what any one of them requires, so please correct me if I'm wrong) are:
Some of those are not obvious and will need a separated discussion, but let's not distract from the main topic for now. I think that the list will look somewhat like this.
On 2/14/20 12:43 PM, Esme Povirk (they/them) wrote:
Options for distributing dependencies: * Distribute headers and libs in a tarball built from wine-pedeps. The binaries would be part of an addon, also built from wine-pedeps.
I personally don't like this. I think there's value in (1) not requiring the user to install libraries they don't need (like, say, libkrb5),
I think that fully functional Wine is the right default. If you think about addons as a part of Wine ecosystem, it's not much different from requiring the user to install jscript.dll even if they don't need it.
(2) not prompting for every add-on (personally I feel that two is already too many),
Sure, it would be nice to improve user experience with Wine addons. The reason I'm not motivated to work on it myself is that it's really just a fallback. If distro does its job well (and I think most of distros do), users don't need that anyway. IMHO we should concentrate on making it easy for distros to ship addons themselves, but fallback improvements are welcomed as well.
(3) letting the distribution choose what to distribute by default.
Sure, it should be possible to disable things at build time. Just like it's already possible to disable jscript.dll at build time.
* Rely on the host distribution to package mingw headers, libs, and binaries for the dependencies we need. The binaries would then be distributed with Wine. Fedora is the only distribution I'm aware of that has an extensive library of mingw packages. To make this easier on other distros, I could maintain wine-pedeps as an alternative source of headers, libraries, and binaries at build time. The trouble with this approach is that it's not clear how we can keep Wine's libraries independent from the application.
This was the idea that I proposed when the discussion came up internally. It strikes me as the architecturally "correct" thing to do. We're essentially building some of our files with a different host architecture, so just as we'd link separately to x86_64-pc-linux-gnu libraries and i686-pc-linux-gnu libraries, so too would we link separately to x86_64-w64-mingw32 libraries (as well as i686-w64-mingw32).
Yes, some distros provide such packages and it's nice. I happened to contribute to that on mingw-w64 side. The purpose of those packages is, however, different than what we need for Wine. It's meant to provide a nice way for cross compiling projects with open source dependences. And once you cross compile such an application and want to ship it, you need to copy required DLLs or statically link dependencies. It's a very different goal than assembling Windows distro that we need.
What you propose is essentially forcing packagers to assembly and Windows (Wine) distro and I don't like that. Linux distros should concentrate on providing good native userspace and it's Wine's responsibility to use that to provide Windows-compatible environment. The fact that we will leverage PE build of some project should be something that packager can control, but he should not be forced to take care and maintain 20 packages only for Wine.
We definitely don't want such Wine dependences to be used in place of application provided DLLs with the same name. Renaming the library in build time seems like way cleaner way than any runtime heuristics that this thread led to.
Another thing where your solution can become a nightmare is ABI incompatibility. We don't control the dependencies and some of them are not ABI stable. We just need to accept it and make sure our solution takes that into account. So for example, when something like this happens (I don't think this is an isolated example, but I happen to know about it and it's hilarious: they had an incompatible MSVC and mingw ABI, so decided to break and uglify mingw ABI to follow MSVC and hack around a problem that I believe is imaginary):
https://bugzilla.libsdl.org/show_bug.cgi?id=4771
What I propose is to have a solution where we just bump just one constant in one repo, which would make Wine use wine-sdl2-2.dll instead of wine-sdl2-1.dll and that's it. All distros are be happy.
What you propose is that now tens of distros would need to notice the problem and express it in their package dependences. It means tens of action tickets spread around the ecosystem and a few Wine NOTOURBUG bugs reported upstream. And all of that work is just to provide a library that's otherwise internal to Wine anyway.
I don't know what we'd do about Mac, but along the same lines, I'd think we'd want to extend/analogize the current solution regarding bitness to the whole architecture triple.
Well Mac would be even easier to target than it is now if we provide right tools (because some of those dependences are tricky on Mac already). And if some packagers really insist on splitting those packages, they can still do that. with what I proposed. They can use our build system as a 'driver' or do it manually, that's all fine. We'd be not forcing one way or another.
So let me give a few examples of different choices packagers can do. I will call wine-core Wine as we know today (which will not change architecturally) and wine-meta-distro the proposed separated repo for dependencies build system allowing a simple and straightforward way to bootstrap complete Wine+dependences package, depending only on cross compilation tools.
1. Not care, provide just wine-core package (addon headers are just a build time dependency). Wine will download and cache addons in runtime.
2. Use wine-core the way distro currently do to provide Wine package. Use wine-meta-distro to build Wine addons as one but separated package.
3. Use wine-meta-distro to produce both a fresh wine-core and wine addons package in a single step for a single package.
4. Like 2 or 3, but distro wants some customization itself. They can do that, for example, by pointing wine-meta-distro to its own version of relevant package, pass additional config options or do simple source code modifications.
5. Split Wine into 20 packages. A distro can use wine-meta-distro to do build config stuff, so that they don't need to change their scripts every time something changes in any of dependent packages, but otherwise treat every dependency as a separated package.
6. Split into 20 packages and deal with them without wine-meta-distro packages. This will require packagers to redo all the work that packagers choosing 1-5 will be getting for free.
Your proposal seems to be similar to something between 5 and 6. I don't expect packagers to be thrilled with that idea if they have 2-4 as alternatives, by the key point is: let's give them a choice.
As a side note, a different and separated distro packages for, say, libxml2 built with winegcc could also have some use cases. That would be something potentially useful for cross compiling an application using winegcc, but not something for Wine to use internally.
Cheers,
Jacek