On 15.04.2020 21:29, Zebediah Figura wrote:
So I think I may have misunderstood or lost sight of goals somewhere along the way, let me try to review.
I think it makes the most sense to use one or more separate distribution packages for our PE dependencies, rather than distributing them as part of Wine, statically linking, or distributing them using our Win32-based addons system. I'm not sure just from reading this thread there's a general consensus regarding that, but I'm not sure I've seen anyone argue against it either.
Separately, I think it makes sense for Wine's libraries—seeing as they *are* (relatively, and mostly) high-level—to use MinGW libraries the same as any other MinGW-compiled program would.
It's not clear to me that this implies that we reverse the direction we're currently taking and try to make more use of the MinGW CRT, headers, and/or libraries. (I can understand the CRT part, though even then I have to wonder if there are differences that will matter.)
If it does, I'd like to understand why that's not possible, because it seems to me that we gave up on MinGW after they objected to taking the entire mingw-w64 project under the umbrella of Wine
Let's make it clear: there is no give up. Cooperation goes on, just like it did before. It just didn't change. We continue to use mingw to build Wine, we just don't use a few static libraries that we didn't need anyway. To give you an example for the context, look at another prominent mingw-w64: Cygwin. For obvious reason they are interested in Win32 platform and use mingw-w64 for platform headers and importlibs. But they have their own ABI, for which they provide their own C runtime libraries. mingw-w64 allows that, the build system has special features to make such things possible. Cygwin also offers mingw-w64 in a form of a cross compiler, which cross compiles from Cygwin ABI as a host to mingw ABI target (obviously using mingw crt). It's an extreme example why "PE is all the same" in not true. That analogy also applies to Wine in a more delicate way. msvcrt/ucrtbase implements a crt for msvc ABI, which is very similar, but different than mingw ABI. Wine needs to provide msvcrt anyway for obvious reasons and that's about all we need as for crt. mingw uses ABI that is slightly different, which has quite a few implications for crt. It's a mixture of static libs and msvcrt importlib with some features that are less than obvious. For example mingw has a real long double which on msvc is the same as double. For that reason, mingw-w64 crt can't use msvcrt long double functions, but instead provides its own implementations of relevant functions in a static lib. If you want full support for long double, you may even choose to use *printf functions provided by mingw-w64 that support long double formats (and also other standard compliance, which was handy at a time, but not so much with ucrt around). long double is obviously not a blocker, it's just an example to give you high level understanding what mingw-w64 crt really is and why. Now, look what fully using mingw-w64 for crt means part by part: - mingw-w64 crt headers. Wine has its own msvcrt headers that could not be blindly just replaced. For example we need those headers to work on non-mingw compilers (like for ELF builds). We'd need make mingw-w64 worry about platform it otherwise doesn't need to care. It's doable, but it's not clear if worth it. - mingw-w64 msvcrt importlib As explained at importlib level a lot of functions are implemented in static lib instead of importing from DLL. We provide importlib in form of .spec file anyway. - standard compatibility stuff mingw-w64 has a number of features providing standard compliance in places where msvcrt did bad job. It was handy in the past for msvcrt version that were rather bad. It's not so useful on top of ucrt. - compatibility stuff A lot of functions are in static lib for compatibility with old msvcrt versions that didn't provide them. A lot of those are just manual delay loads with some static fallbacks. We don't need those. - more advanced startup files, a few features that really belong to static lib like atexit We don't need any of those for Wine itself. We may need some in the future. I hope that answers your question about crt part. If you still think there is a problem, please be more concrete. That said, there is definitely room for improvements. I happen to be the one maintaining sharing that actually happens for years (which is why I find it ironic that you complain to me about lack of sharing) and I plan to continue doing so. Let me use a chance to raise awareness in case someone wants to help. mingw-w64 semi-automatically imports a number of headers and widl from Wine tree. On mingw-w64 side there is assumption that for imported stuff, Wine review replaces mingw-w64 review process. mingw-w64 has two import scripts that are really basic and trivial to maintain scripts doing the job. If you'd prefer to do it differently, patches welcomed. We could share a lot more platform headers, but it's a real work. In case a header is missing in mingw-w64, it's a matter of testing and making sure all its dependences are fine. If it's already duplicated between projects, then deduplication needs a review to make sure that one of copies is complete enough to replace the other by sending a patch to Wine. Once it's in Wine, send a patch to import script to mingw-w64. From this moment, both repos will stay synced. If you hypothetically want fully merged platform headers in the future, this is the way to do it incrementally. And it's already there. widl could also use some work. The most notable example that sees demand on mingw-w64 side is better winrt support. If you really want to help mingw-w64 and Wine, I think that contributing to any of above is a good way to do it.
Now, on the assumption that using mingw-w64-compiled PE libraries is a reasonable thing to do...
On 4/13/20 11:38 AM, Jacek Caban wrote:
So let me give a few examples of different choices packagers can do. I will call wine-core Wine as we know today (which will not change architecturally) and wine-meta-distro the proposed separated repo for dependencies build system allowing a simple and straightforward way to bootstrap complete Wine+dependences package, depending only on cross compilation tools.
1. Not care, provide just wine-core package (addon headers are just a build time dependency). Wine will download and cache addons in runtime.
2. Use wine-core the way distro currently do to provide Wine package. Use wine-meta-distro to build Wine addons as one but separated package.
3. Use wine-meta-distro to produce both a fresh wine-core and wine addons package in a single step for a single package.
4. Like 2 or 3, but distro wants some customization itself. They can do that, for example, by pointing wine-meta-distro to its own version of relevant package, pass additional config options or do simple source code modifications.
5. Split Wine into 20 packages. A distro can use wine-meta-distro to do build config stuff, so that they don't need to change their scripts every time something changes in any of dependent packages, but otherwise treat every dependency as a separated package.
6. Split into 20 packages and deal with them without wine-meta-distro packages. This will require packagers to redo all the work that packagers choosing 1-5 will be getting for free. Your proposal seems to be similar to something between 5 and 6. I don't expect packagers to be thrilled with that idea if they have 2-4 as alternatives, by the key point is: let's give them a choice.
As a side note, a different and separated distro packages for, say, libxml2 built with winegcc could also have some use cases. That would be something potentially useful for cross compiling an application using winegcc, but not something for Wine to use internally.
I think 2/4 are referring to the same thing (at any rate, I see no reason not to give distributions the choice). I don't think I fully understand how 3 differs from 2/4, or how 6 differs from 5. Do you mean by introducing a meta-package which only exists for dependency purposes?
Anyway, if depending on mingw-w64 really is a dead end, I guess either 4 or 5 seems reasonable to me, though it's less clear that distributions actually would prefer 4. In particular, I notice:
* Suppose you want the versions of the PE libraries you ship to match the ELF libraries your distribution already ships. You can change the sources without too much difficulty, but how do you label the resulting package with regard to version? Naming it after the Wine version isn't great, especially if you're going to update those packages asynchronously...
It's not very important implementation detail that we may decide once we have it sorted out on higher level.
* If a distribution (or even WineHQ) wants to update one of our 20+ packages, do they/we have to ship the whole wine-meta-distro again? (Maybe this isn't a problem, if the packages in question support partial updates...)
Well, just like we'd update jscript.dll in Wine installation even if it didn't change since last update.
Ultimately I think this is something that actual distributors would have to answer, and I think we should probably bring some in on this thread.
If on the other hand we can depend on mingw-w64 libraries, I think we almost have to pick 5 over 4, for the simple reason that it's already done that way. There's an existing demand for mingw-w64 libraries which are distributed separately. Fedora already ships them. Arch doesn't (in fact, Arch doesn't even ship the mingw-w64 compiler), but they exist in the AUR as separate packages, and I myself use a third-party repository that ships mingw-w64 library blobs. (That's how I was able to write that msxml3 patch, in fact.) Debian/Ubuntu don't ship very many, but they do ship libz, and also development versions of a few other libraries.
Let me give a high level summary how I see it. I'm not attached to what I proposed, but here is how I can see the situation: We're going to be shipping PE build ourselves one way or another (all solutions assume that so far). Whatever form that will take, we should make sure that it's something reusable for others. By ensuring that whatever we do is easy to reuse, we can make sure that forks and distros have the ability to do the same without duplicating efforts. I think that the solution should be portable. Limiting it to a distro or a set of distros is not what I consider as portable. We will want to provide distro packages, just like we provide Wine packages. But I think that actual maintenance should be happening in platform-independent place. Just like Wine has no bits specific to any package manager in its sources. It seems to me that autoconf and make provide tools to achieve that. Jacek