[Bug 54940] New: Import library generation is much slower without dlltool
https://bugs.winehq.org/show_bug.cgi?id=54940 Bug ID: 54940 Summary: Import library generation is much slower without dlltool Product: Wine Version: 8.0-rc1 Hardware: x86-64 OS: Linux Status: NEW Keywords: regression Severity: normal Priority: P2 Component: build-env Assignee: wine-bugs(a)winehq.org Reporter: z.figura12(a)gmail.com CC: rbernon(a)codeweavers.com Regression SHA1: 103eb831564decc9812239f454ed1cc40714c0a1 Distribution: --- E.g.: elizabeth(a)ledyba:~/git/wine64$ time tools/winebuild/winebuild -w --implib -o dlls/msi/x86_64-windows/libmsi.a --without-dlltool -b x86_64-w64-mingw32 --export ../wine/dlls/msi/msi.spec real 0m1.509s user 0m0.575s sys 0m0.481s elizabeth(a)ledyba:~/git/wine64$ time tools/winebuild/winebuild -w --implib -o dlls/msi/x86_64-windows/libmsi.a -b x86_64-w64-mingw32 --export ../wine/dlls/msi/msi.spec real 0m0.042s user 0m0.017s sys 0m0.025s -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54940 --- Comment #1 from Zeb Figura <z.figura12(a)gmail.com> --- Annoyingly, simply reverting is not an option: after 402d50d5 and other commits which switch to using proper dllimports, anything which uses a delay import crashes, probably due to [1]. [1] https://sourceware.org/bugzilla/show_bug.cgi?id=14339 -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54940 --- Comment #2 from Rémi Bernon <rbernon(a)codeweavers.com> --- I don't see what we can do about it, we need separate objects so that only the used symbols are pulled in and there doesn't seem to be a way to assemble multiple files into separate objects in one invocation. It also doesn't look realistic to write the archive directly, delay imports use small thunks and we would have to hardcode their binary form, which doesn't look very appealing. If the time it takes really is an issue, I can only suggest to switch to a more compatible toolchain, like LLVM, or maybe use its linker instead of GNU ld, but I don't think import libs are rebuilt so often. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54940 --- Comment #3 from Zeb Figura <z.figura12(a)gmail.com> --- This definitely has made iterative development slower for me. Anyway, I can think of at least two possible solutions for this: * Fix GNU dlltool, and allow using the system dlltool if it works correctly. * Take our cues from GNU dlltool, and (optionally) link directly to libbfd in order to create a single archive with separate sections. This would require making winebuild GPL3, but according to my understanding that doesn't mean that Wine itself needs to be GPL3. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54940 Jacek Caban <jacek(a)codeweavers.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jacek(a)codeweavers.com --- Comment #4 from Jacek Caban <jacek(a)codeweavers.com> --- Ideally, importlibs should not need assembler at all. There is a separated format for COFF import libs, which is simple enough that we could just generate it ourselves (ideally dlltool would use that instead; llvm-dlltool, llvm-lib and MSVC lib.exe do). When such import libs are used, it's linker's responsibility to generate thunks, importlibs shouldn't contain any actual code. While the above would probably be fine with ld.bfd for regular imports, it wouldn't work for delay load imports. ld.bfd just doesn't support delay loads at all, that's why the hack in dlltool exists in the first place. If someone is willing to work on improving binutils, I suggest starting with linker support for delay loads and getting rid of dependency on the hack. BTW, for ELF target binutils developers decided that it's better to reimplement the linker in form of gold instead of trying to improve ld.bfd. The same is likely true for COFF targets and I see little point in reimplementing it again when lld-link exists. GCC also supports -fuse-ld=lld, it should be possible to make it work that way. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54940 --- Comment #5 from Zeb Figura <z.figura12(a)gmail.com> --- (In reply to Jacek Caban from comment #4)
BTW, for ELF target binutils developers decided that it's better to reimplement the linker in form of gold instead of trying to improve ld.bfd. The same is likely true for COFF targets
Huh? ld is still seeing active development; in fact both ld and bfd have an order of magnitude more development than gold. Maybe it's a good idea to implement another gold for COFF just for speed, but I don't see why that's relevant to this issue at all.
and I see little point in reimplementing it again when lld-link exists. GCC also supports -fuse-ld=lld, it should be possible to make it work that way.
I'm probably getting unnecessarily annoyed, but for some reason it kind of irks me when someone does a hostile fork or reimplementation (with a harmfully permissive license, no less), then I hear claims that there's no reason to use the original project, regardless of the fact that it still exists and sees active development. Especially when lld has some severe problems in terms of size and memory usage. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54940 --- Comment #6 from Jacek Caban <jacek(a)codeweavers.com> --- (In reply to Zeb Figura from comment #5)
Huh? ld is still seeing active development;
Yes, it still exists and yes, it's still slow and memory hungry.
I'm probably getting unnecessarily annoyed, but for some reason it kind of irks me when someone does a hostile fork or reimplementation (with a harmfully permissive license, no less), then I hear claims that there's no reason to use the original project, regardless of the fact that it still exists and sees active development.
I never said there's no reason to use the original project. I think we should allow user to choose their preferred linker by supporting both of them. I believe that competition is good for users. We, as toolchain users, are in a better position having the choice than we'd be if we were stuck with a single provider, no matter which one. My comment was about spending my own time on fixing ld.bfd itself, which is a very different thing than using it (and different than hacking Wine code to deal with its limitations). To elaborate more on that, for me the performance is a deal breaker when it comes to the linker of my first choice (although, since I generally contribute to mingw ecosystem, I have various configuration in my system and test things with all of them). When I ported wine-gecko to llvm-mingw, linking time went down from a few minutes to a few seconds and memory pressure was significantly reduced. When working on wine-gecko, this is a life quality improvement that's hard to negate and it's not realistic to completely fix as a side project. Even if other missing features in ld.bfd would be implemented (like delay load, PDBs or aarch64 support), it would still not be good enough for my own use case. Others may have different priorities. Performance, for example, doesn't matter much when compiling Wine itself, because Wine builds relatively small executables. I'd be more than happy to see binutils being more competitive. That's why I wrote the comment, to give some feedback where to look at. To summarize, if someone is interested in working on it: - If you just want to save a few seconds of Wine build time, fix dlltool. - If you additionally want to meaningfully improve binutils-based mingw toolchain, implement delay load in the linker. - In any case, it's not a Wine bug. Also, I think that calling LLVM a hostile reimplementation is unfair. I guess you're referring to the fact that clang frontend is more-or-less compatible with gcc. I'd say that making porting between compilers easier is a good thing. Why would you intentionally make porting harder? Simplifying porting existing code base is a perfectly valid design choice in my opinion.
Especially when lld has some severe problems in terms of size and memory usage.
Huh, are you sure you not mixed linkers in that statement? Absurdly high memory usage is the reason I sometimes avoid using ld.bfd even on ELF target (where ld.bfd is my default distro linker otherwise, FWIW). I'm speaking from the actual experience, do you? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54940 --- Comment #7 from Zeb Figura <z.figura12(a)gmail.com> --- (In reply to Jacek Caban from comment #6)
I'm probably getting unnecessarily annoyed, but for some reason it kind of irks me when someone does a hostile fork or reimplementation (with a harmfully permissive license, no less), then I hear claims that there's no reason to use the original project, regardless of the fact that it still exists and sees active development.
I never said there's no reason to use the original project. I think we should allow user to choose their preferred linker by supporting both of them. I believe that competition is good for users. We, as toolchain users, are in a better position having the choice than we'd be if we were stuck with a single provider, no matter which one.
I'm sorry, I overreacted. I suppose I misinterpreted your statement about "little point in reimplementing [ld] when lld-link exists".
Also, I think that calling LLVM a hostile reimplementation is unfair. I guess you're referring to the fact that clang frontend is more-or-less compatible with gcc. I'd say that making porting between compilers easier is a good thing. Why would you intentionally make porting harder? Simplifying porting existing code base is a perfectly valid design choice in my opinion.
My understanding was that LLVM is (was?) a hostile reimplementation of gcc specifically, not just a compatible replacement. However that's only based on hearsay and I can't even point to any specific statement.
Especially when lld has some severe problems in terms of size and memory usage.
Huh, are you sure you not mixed linkers in that statement? Absurdly high memory usage is the reason I sometimes avoid using ld.bfd even on ELF target (where ld.bfd is my default distro linker otherwise, FWIW). I'm speaking from the actual experience, do you?
Yes. I remember at least one instance of LLVM specifically running out of memory with a large program when linking. Also, while it's debatable whether it's related, LLVM has terrible problems with the size of its own binaries. It's something I notice whenever I need to debug out-of-memory conditions. libLLVM on this machine is an enormous 132 MiB, and if you count its nontrivial dependencies (including libxml2, libstdc++, libicudata/libicuuc) it's 187 MiB. (Compare libgccjit which is a mere 33 MiB (35 MiB with dependencies), less than a fifth of that. This doesn't inhere to a compiler; it's just that LLVM doesn't care about memory usage.) -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=54940 Alexandre Julliard <julliard(a)winehq.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |NOTOURBUG --- Comment #8 from Alexandre Julliard <julliard(a)winehq.org> --- Resolving as NOTOURBUG then, this should be fixed in binutils. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
participants (1)
-
WineHQ Bugzilla