[Bug 53918] New: Building with external PE libraries fails when static libraries are also installed

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53918

Bug ID: 53918
           Summary: Building with external PE libraries fails when static
                    libraries are also installed
           Product: Wine
           Version: 7.20
          Hardware: x86-64
                OS: Linux
            Status: NEW
          Keywords: regression
          Severity: normal
          Priority: P2
         Component: build-env
          Assignee: [e-mail]
          Reporter: [e-mail]
   Regression SHA1: 6a912649188a765d0c58e8b26e093fb0c31b3e3d
      Distribution: ---

The basic problem is that winegcc, when passed e.g.
"-L/usr/x86_64-w64-mingw32/lib -lvkd3d", looks up that library in the search
path, finds libvkd3d.a, and tries to link to it.

The mingw-w64 linking process, at least for vkd3d (but presumably others as
well), provides both "libvkd3d.a" and "libvkd3d.dll.a". The former is a static
library, and the latter is essentially an import library for libvkd3d-1.dll.
Normal mingw-w64 gcc, when passed -lvkd3d, links to the latter, according to
the search path specified in [1] s.v. "direct linking to a dll". winegcc does
not know about .dll.a and hence finds the static library first.

This worked before 6a912649 because winegcc would try to find a library ending
in ".cross.a", fail to find *any* vkd3d libraries, and just leave the linker
argument as "-lvkd3d".

Given this, an obvious solution is to teach winegcc about the ld linker path.

However, I find myself asking—why are we trying to rewrite the command line at
all? Why can't we just let mingw-w64 gcc work by itself? winegcc is, naturally,
devoid of any comments, and as far as I can tell the code doesn't actually *do*
anything that depends on the library type or path, so I started looking in the
git history, but unfortunately was only partly able to answer the question.
Here is what I found:

* c7a210bc2f27 forces the use of library paths if -mno-cygwin is set (in
addition to some other conditions). The commit message specifies that we *can*
do this (due to our use of -nodefaultlibs), but doesn't specify why we *need*
to do this. Jacek, can you by chance explain?

* 33147c947500 forces the use of library paths (sort of) if lib_suffix is set,
which makes sense—gcc won't be able to find any libraries if we are overriding
lib_suffix.

* 870d490eecdb forces the use of library paths for all winelib DLLs, so that
"-lfoo" links to "foo/foo.dll". This makes sense, but is also now obsolete—our
makefiles pass the direct paths. Or do we want to support winelib users outside
of wine including from the build tree?

* 006ec80dd5c1 introduces the logic in the first place, forcing the use of
library paths for static libraries. This was written by Dimitrie O. Paun, with
the comment "For static libs (.a) we need to pass the actual filename to
winebuild, not a -l switch." No further explanation is given.

Alexandre, can you please shed light on this?

[1] https://sourceware.org/binutils/docs-2.39/ld/WIN32.html