[PATCH v2 0/3] MR6715: Support separated debug information for PE images.
This serie is a first attempt to provide a solution to bug 57308 [1]. Basically, Arch distro is looking for Wine support in distributing debug information for Wine PE modules separated from image. A quick overview of distro (Debian, Fedora, Arch) situation i (I may have missed a couple of items), but: - no distro ship separate debug information for PE modules (they do it for ELF ones though) - at best, they ship unstripped PE modules. IMO, we need to help them pave the way to what should be the best expected situation (in Wine and wrt distros) to support separate debug information for PE images. First, I believe we need to use a (solid) key to bind the stripped image and the file holding the debug information. In ELF format, this is typically done by inserting a .note.gnu.build-id section (in both stripped image and debug information file) which contains the key. This key is used to either search in fixed location (eg /usr/lib/.debug), or as key to download the debug information from a remote server (debuginfod). The key is generally made of a hash of (some sections of) the image file. This allows (esp. in the case of downloading the debug information from a debugger) to discriminate between several versions of the same module. I think we should target this kind of solution instead of storing the debug information alongside the module (usually done as ntdll.dll.debug). This wouldn't let debuggers download the expected debug info upon request. Fortunately, gcc and clang provide a build-id option (when generating PE images) which stores this key in the PE image. What this serie does: - fixes & improves dbghelp in the support of the build-id information from gcc/mingw & clang, - improves Wine's configure script so that clang can be used to generate dwarf debug information and build-id information in PE builds, - adds a couple of additional search locations for debug information files, It takes the following assumptions for the storage: - debug information files for PE images are stored in same hierarchy as debug information files for ELF images (eg /usr/lib/.debug) - naming of such files is done as ELF, ie by using the hex string generated from the binary blob of the key. This is *NOT* the usual way how the GUID will be printed (because the first 3 integers of the GUID are stored in little-endian format). (Note: I had to make a choice here. I don't have strong arguments one way or the other, but we need to agree on the convention here). What distro should do to support separate debug info: - configure Wine with --enable-build-id - adapt at least the packaging tool to extract the key out of unstripped image [2], - if binutils isn't compiled with PE support (this is apparently the Arch case), adapt the scripts to use x86_64-objcopy instead of objcopy (and friends). I marked this MR as draft for now, as: - I still need to polish some items (like the default location for looking up debug information files), - wait for feedback on this proposal (it does make a couple of assumptions that need sharing IMO), Further steps: - the steps above are mainly targetted to have a solution for packaged download of debug information files, - this should be integrated in debuginfod (server side: ingestion of PE files and client side: ensure gdb, lldb are compatible; implement debuginfod in winedbg). Feed back welcomed! [1]: https://bugs.winehq.org/show_bug.cgi?id=57308 [2]: The least invasive readelf replacement (assuming binutils is compiled with PE support). (otherwise, use x86_64-objdump) ``` BUILDID=`objdump -p "$INPUT" | sed -n "/RSDS signature / {s/.*signature //; s/ age.*//p; q;}"` BUILDID=${BUILDID:6:2}${BUILDID:4:2}${BUILDID:2:2}${BUILDID:0:2}${BUILDID:10:2}${BUILDID:8:2}${BUILDID:14:2}${BUILDID:12:2}${BUILDID:16} ``` Additional note: gcc/mingw puts the desired debug entry inside a dedicated section (.buildid), while clangs keeps it in .rdata (as msvc does). So, this invalidates any attempt to get information from the .buildid section (as stated in bug report). -- v2: configure.ac: Improve clang configuration for dwarf as cross debug format. dbghelp: Search debug info with buildid for RSDS debug entry w/o filenames. dbghelp: Extend search for buildid in system directories. https://gitlab.winehq.org/wine/wine/-/merge_requests/6715
From: Eric Pouech <epouech(a)codeweavers.com> Search for files <buildid> and <buildid>.debug Fedora (at least) now put the debug info here (instead of /usr/lib/debug/.build). So search both directories. Signed-off-by: Eric Pouech <epouech(a)codeweavers.com> --- dlls/dbghelp/module.c | 93 ++++++++++++++++++++++++------------------- 1 file changed, 52 insertions(+), 41 deletions(-) diff --git a/dlls/dbghelp/module.c b/dlls/dbghelp/module.c index 4b4dbbaf595..26a43be875b 100644 --- a/dlls/dbghelp/module.c +++ b/dlls/dbghelp/module.c @@ -664,69 +664,80 @@ static WCHAR* append_hex(WCHAR* dst, const BYTE* id, const BYTE* end) return dst; } -/****************************************************************** - * image_locate_build_id_target - * - * Try to find the .so file containing the debug info out of the build-id note information - */ -static struct image_file_map* image_locate_build_id_target(const BYTE* id, unsigned idlen) +static BOOL image_locate_build_id_target_in_dir(struct image_file_map *fmap_link, const BYTE* id, unsigned idlen, const WCHAR *from) { - struct image_file_map* fmap_link = NULL; - DWORD sz; - WCHAR* p; - WCHAR* z; + size_t from_len = wcslen(from); + BOOL found = FALSE; + WCHAR *p, *z; - fmap_link = HeapAlloc(GetProcessHeap(), 0, sizeof(*fmap_link)); - if (!fmap_link) return NULL; - - p = malloc(sizeof(L"/usr/lib/debug/.build-id/") + - (idlen * 2 + 1) * sizeof(WCHAR) + sizeof(L".debug")); - if (!p) goto fail; - wcscpy(p, L"/usr/lib/debug/.build-id/"); - z = p + wcslen(p); - if (idlen) + if ((p = malloc((from_len + idlen * 2 + 1) * sizeof(WCHAR) + sizeof(L".debug")))) { + memcpy(p, from, from_len * sizeof(WCHAR)); + z = p + from_len; z = append_hex(z, id, id + 1); if (idlen > 1) { *z++ = L'/'; z = append_hex(z, id + 1, id + idlen); } + *z = L'\0'; + TRACE("checking %s\n", debugstr_w(p)); + found = image_check_debug_link_gnu_id(p, fmap_link, id, idlen); + if (!found) + { + wcscpy(z, L".debug"); + TRACE("checking %s\n", debugstr_w(p)); + found = image_check_debug_link_gnu_id(p, fmap_link, id, idlen); + } + free(p); } - wcscpy(z, L".debug"); - TRACE("checking %s\n", debugstr_w(p)); + return found; +} - if (image_check_debug_link_gnu_id(p, fmap_link, id, idlen)) - { - free(p); +/****************************************************************** + * image_locate_build_id_target + * + * Try to find an image file containing the debug info out of the build-id + * note information. + */ +static struct image_file_map* image_locate_build_id_target(const BYTE* id, unsigned idlen) +{ + struct image_file_map* fmap_link; + DWORD sz; + + if (!idlen) return NULL; + + if (!(fmap_link = HeapAlloc(GetProcessHeap(), 0, sizeof(*fmap_link)))) + return NULL; + if (image_locate_build_id_target_in_dir(fmap_link, id, idlen, L"/usr/lib/debug/.build-id/")) + return fmap_link; + if (image_locate_build_id_target_in_dir(fmap_link, id, idlen, L"/usr/lib/.build-id/")) return fmap_link; - } sz = GetEnvironmentVariableW(L"WINEHOMEDIR", NULL, 0); if (sz) { - z = realloc(p, sz * sizeof(WCHAR) + - sizeof(L"\\.cache\\debuginfod_client\\") + - idlen * 2 * sizeof(WCHAR) + sizeof(L"\\debuginfo") + 500); - if (!z) goto fail; - p = z; - GetEnvironmentVariableW(L"WINEHOMEDIR", p, sz); - z = p + sz - 1; - wcscpy(z, L"\\.cache\\debuginfod_client\\"); - z += wcslen(z); - z = append_hex(z, id, id + idlen); - wcscpy(z, L"\\debuginfo"); - TRACE("checking %ls\n", p); - if (image_check_debug_link_gnu_id(p, fmap_link, id, idlen)) + WCHAR *p, *z; + p = malloc(sz * sizeof(WCHAR) + + sizeof(L"\\.cache\\debuginfod_client\\") + + idlen * 2 * sizeof(WCHAR) + sizeof(L"\\debuginfo") + 500); + if (p && GetEnvironmentVariableW(L"WINEHOMEDIR", p, sz) == sz - 1) { + BOOL found; + + wcscpy(p + sz - 1, L"\\.cache\\debuginfod_client\\"); + z = p + wcslen(p); + z = append_hex(z, id, id + idlen); + wcscpy(z, L"\\debuginfo"); + TRACE("checking %ls\n", p); + found = image_check_debug_link_gnu_id(p, fmap_link, id, idlen); free(p); - return fmap_link; + if (found) return fmap_link; } } TRACE("not found\n"); -fail: - free(p); + HeapFree(GetProcessHeap(), 0, fmap_link); return NULL; } -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/6715
From: Eric Pouech <epouech(a)codeweavers.com> Signed-off-by: Eric Pouech <epouech(a)codeweavers.com> --- dlls/dbghelp/dbghelp_private.h | 2 ++ dlls/dbghelp/module.c | 20 ++++++++++++++++++-- dlls/dbghelp/pe_module.c | 24 ++++++++++++++++++++++++ 3 files changed, 44 insertions(+), 2 deletions(-) diff --git a/dlls/dbghelp/dbghelp_private.h b/dlls/dbghelp/dbghelp_private.h index d8761bd599a..8def6a7dae6 100644 --- a/dlls/dbghelp/dbghelp_private.h +++ b/dlls/dbghelp/dbghelp_private.h @@ -788,6 +788,8 @@ extern BOOL pe_unmap_directory(struct module* module, int dirno, const c extern DWORD pe_get_file_indexinfo(void* image, DWORD size, SYMSRV_INDEX_INFOW* info); extern const BYTE* pe_lock_region_from_rva(struct module *module, DWORD rva, DWORD size, DWORD *length); extern BOOL pe_unlock_region(struct module *module, const BYTE* region); +struct image_file_map; +extern BOOL pe_has_buildid_debug(struct image_file_map *fmap, GUID *guid); /* source.c */ extern unsigned source_new(struct module* module, const char* basedir, const char* source); diff --git a/dlls/dbghelp/module.c b/dlls/dbghelp/module.c index 26a43be875b..61331c4c5b2 100644 --- a/dlls/dbghelp/module.c +++ b/dlls/dbghelp/module.c @@ -527,6 +527,7 @@ static BOOL image_check_debug_link_gnu_id(const WCHAR* file, struct image_file_m { struct image_section_map buildid_sect; DWORD read_bytes; + GUID guid; HANDLE handle; WCHAR *path; WORD magic; @@ -545,6 +546,9 @@ static BOOL image_check_debug_link_gnu_id(const WCHAR* file, struct image_file_m ret = elf_map_handle(handle, fmap); CloseHandle(handle); + if (ret && pe_has_buildid_debug(fmap, &guid)) + return TRUE; + if (ret && image_find_section(fmap, ".note.gnu.build-id", &buildid_sect)) { const UINT32* note; @@ -557,12 +561,13 @@ static BOOL image_check_debug_link_gnu_id(const WCHAR* file, struct image_file_m { if (note[1] == idlen && !memcmp(note + 3 + ((note[0] + 3) >> 2), id, idlen)) return TRUE; - WARN("mismatch in buildid information for %s\n", debugstr_w(file)); } } image_unmap_section(&buildid_sect); image_unmap_file(fmap); } + if (ret) + WARN("mismatch in buildid information for %s\n", debugstr_w(file)); return FALSE; } @@ -849,7 +854,18 @@ BOOL image_check_alternate(struct image_file_map* fmap, const struct module* mod struct image_file_map* fmap_link = NULL; /* if present, add the .gnu_debuglink file as an alternate to current one */ - if (image_find_section(fmap, ".note.gnu.build-id", &buildid_sect)) + if (fmap->modtype == DMT_PE) + { + GUID guid; + + if (pe_has_buildid_debug(fmap, &guid)) + { + /* reorder bytes to match little endian order */ + fmap_link = image_locate_build_id_target((const BYTE*)&guid, sizeof(guid)); + } + } + /* if present, add the .note.gnu.build-id as an alternate to current one */ + if (!fmap_link && image_find_section(fmap, ".note.gnu.build-id", &buildid_sect)) { const UINT32* note; diff --git a/dlls/dbghelp/pe_module.c b/dlls/dbghelp/pe_module.c index 75060e2dfa6..bd3d6000535 100644 --- a/dlls/dbghelp/pe_module.c +++ b/dlls/dbghelp/pe_module.c @@ -1036,3 +1036,27 @@ DWORD pe_get_file_indexinfo(void* image, DWORD size, SYMSRV_INDEX_INFOW* info) } return msc_get_file_indexinfo(image, dbg, dirsize / sizeof(*dbg), info); } + +/* check if image contains a debug entry that contains a gcc/mingw - clang build-id information */ +BOOL pe_has_buildid_debug(struct image_file_map *fmap, GUID *guid) +{ + BOOL ret = FALSE; + + if (fmap->modtype == DMT_PE) + { + SYMSRV_INDEX_INFOW info = {.sizeofstruct = sizeof(info)}; + const void *image = pe_map_full(fmap, NULL); + + if (image) + { + DWORD retval = pe_get_file_indexinfo((void*)image, GetFileSize(fmap->u.pe.hMap, NULL), &info); + if ((retval == ERROR_SUCCESS || retval == ERROR_BAD_EXE_FORMAT) && info.age && !info.pdbfile[0]) + { + *guid = info.guid; + ret = TRUE; + } + pe_unmap_full(fmap); + } + } + return ret; +} -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/6715
From: Eric Pouech <epouech(a)codeweavers.com> Signed-off-by: Eric Pouech <epouech(a)codeweavers.com> --- configure.ac | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/configure.ac b/configure.ac index dd80e0c06d8..b721f66255e 100644 --- a/configure.ac +++ b/configure.ac @@ -1004,16 +1004,23 @@ This is an error since --enable-archs=$wine_arch was requested.])]) if test "x$ac_debug_format_seen" = x then case $wine_crossdebug in - *dwarf) WINE_TRY_PE_CFLAGS([-gdwarf-4]) ;; + *dwarf) WINE_TRY_PE_CFLAGS([-gdwarf-4],[:], + [WINE_TRY_PE_CFLAGS([-gdwarf-4 -Wl,-debug:dwarf], + [AS_VAR_APPEND([${wine_arch}_CFLAGS],[" -gdwarf-4"]) + AS_VAR_APPEND([${wine_arch}_LDFLAGS],[" -Wl,-debug:dwarf"])])]) ;; pdb) WINE_TRY_PE_CFLAGS([-gcodeview]) ;; esac fi AS_VAR_SET([${wine_arch}_DEBUG],[$wine_crossdebug]) test "x$enable_werror" != xyes || WINE_TRY_PE_CFLAGS([-Werror]) - test "x$enable_build_id" != xyes || WINE_TRY_PE_CFLAGS([-Wl,--build-id], - [AS_VAR_APPEND([${wine_arch}_CFLAGS],[" -Wl,--build-id"]) - AS_VAR_APPEND([${wine_arch}_LDFLAGS],[" -Wl,--build-id"])]) + if test "x$enable_build_id" = xyes + then + WINE_TRY_PE_CFLAGS([-Wl,--build-id], + [AS_VAR_APPEND([${wine_arch}_LDFLAGS],[" -Wl,--build-id"])], + [WINE_TRY_PE_CFLAGS([-Wl,-build-id], + [AS_VAR_APPEND([${wine_arch}_LDFLAGS],[" -Wl,-build-id"])])]) + fi done -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/6715
Hi, It looks like your patch introduced the new failures shown below. Please investigate and fix them before resubmitting your patch. If they are not new, fixing them anyway would help a lot. Otherwise please ask for the known failures list to be updated. The tests also ran into some preexisting test failures. If you know how to fix them that would be helpful. See the TestBot job for the details: The full results can be found at: https://testbot.winehq.org/JobDetails.pl?Key=149759 Your paranoid android. === debian11b (64 bit WoW report) === user32: win.c:4070: Test failed: Expected active window 000000000409013A, got 0000000000000000. win.c:4071: Test failed: Expected focus window 000000000409013A, got 0000000000000000.
V2 pushed: * removed already committed patches -- https://gitlab.winehq.org/wine/wine/-/merge_requests/6715#note_87971
Alexandre Julliard (@julliard) commented about configure.ac:
if test "x$ac_debug_format_seen" = x then case $wine_crossdebug in - *dwarf) WINE_TRY_PE_CFLAGS([-gdwarf-4]) ;; + *dwarf) WINE_TRY_PE_CFLAGS([-gdwarf-4],[:], + [WINE_TRY_PE_CFLAGS([-gdwarf-4 -Wl,-debug:dwarf], + [AS_VAR_APPEND([${wine_arch}_CFLAGS],[" -gdwarf-4"]) + AS_VAR_APPEND([${wine_arch}_LDFLAGS],[" -Wl,-debug:dwarf"])])]) ;;
This doesn't make much sense, what are you trying to do here? -- https://gitlab.winehq.org/wine/wine/-/merge_requests/6715#note_88352
On Fri Nov 22 08:07:04 2024 +0000, Alexandre Julliard wrote:
This doesn't make much sense, what are you trying to do here? I think it was a workaround for what has been fixed in MR!6851
I'll remove it. Note: this MR was still meant to be in draft status as I'm waiting for testing feedback on Arch side. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/6715#note_88456
But I can't repro here. Could you share your configure options? Maybe I didn't retest with your combination.
Regarding the issue with `-g` and `-gdwarf-4`, I can still reproduce with the AUR package wine-git (see [PKGBUILD](https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=wine-git#n109) for configure options). I can see mingw compilers are invoked with `-g` instead of `-gdwarf-4` in build logs. Complete logs: [wine-git-9.22.r0.g51ccd95c49c-1-x86_64-build.log.gz](/uploads/827938ba7eacbec7d561b0d25ff74163/wine-git-9.22.r0.g51ccd95c49c-1-x86_64-build.log.gz) Here the wine-git package undergoes similar modifications below, while without `-g` -> `-gdwarf-4` changes. ---
dbghelp (with this MR) searches both /usr/lib/debug/.build-id/xxx/yyy and /usr/lib/debug/.build-id/xxx/yyy.debug so something else must be going wrong.
It seems dbghelp stops after it doesn't find debug symbols in /usr/lib/debug/.build-id/xxx/yyy. I ran a test program which crashes in msvcrt.exe and got: ``` 017c:trace:dbghelp:image_locate_build_id_target_in_dir checking L"/usr/lib/debug/.build-id/b7/7e9f19b7d1e8f889b2b790d6116a1d" 017c:trace:dbghelp:image_check_debug_link_gnu_id Located debug information file at L"/usr/lib/debug/.build-id/b7/7e9f19b7d1e8f889b2b790d6116a1d" 017c:trace:dbghelp:pe_load_stabs failed to load the STABS debug info 017c:trace:dbghelp_dwarf:dwarf2_parse Loading Dwarf2 information for L"msvcrt" 017c:trace:dbghelp:image_load_debugaltlink No .gnu_debugaltlink section found for L"msvcrt" 017c:trace:dbghelp:pe_load_dwarf successfully loaded the DWARF debug info ``` Here /usr/lib/debug/.build-id/b7/7e9f19b7d1e8f889b2b790d6116a1d points to /usr/lib/wine/x86_64-windows/msvcrt.dll. Complete wine logs: [winedbg.log](/uploads/5b93c494acfd801b282b8cae3cb0154b/winedbg.log) Test program: [test-debugging-symbols.c](/uploads/cc7979fadadec245c548b49355106383/test-debugging-symbols.c) Here wine is built using wine-git AUR package with the following modifications: ```diff diff --git a/PKGBUILD b/PKGBUILD index e679471..ccdf826 100644 --- a/PKGBUILD +++ b/PKGBUILD @@ -88,6 +88,15 @@ sha256sums=('SKIP' prepare() { rm -rf build-{32,64} mkdir -p build-{32,64} + + cd wine + + # https://gitlab.winehq.org/wine/wine/-/merge_requests/6715 + git cherry-pick -n 3e0eda8986172fe77c3f871100d46f62c9fcd2d6 + git cherry-pick -n 4fb40e04a6d41508beb53a70a70470cc1102a863 + git cherry-pick -n 8283999ec06e5cd8af5fd4fdf6d29765571c6f63 + + autoreconf -ifv } pkgver() { @@ -98,8 +107,8 @@ build() { export CFLAGS+=' -ffat-lto-objects' # apply flags for cross-compilation - export CROSSCFLAGS="${CFLAGS/-Werror=format-security/} -g" - export CROSSCXXFLAGS="${CXXFLAGS/-Werror=format-security/} -g" + export CROSSCFLAGS="${CFLAGS/-Werror=format-security/} -gdwarf-4" + export CROSSCXXFLAGS="${CXXFLAGS/-Werror=format-security/} -gdwarf-4" export CROSSLDFLAGS="${LDFLAGS//-Wl,-z*([^[:space:]])/}" # build wine 64-bit @@ -112,6 +121,7 @@ build() { --with-x \ --with-wayland \ --with-gstreamer \ + --enable-build-id \ --enable-win64 make @@ -125,6 +135,7 @@ build() { --with-x \ --with-wayland \ --with-gstreamer \ + --enable-build-id \ --with-wine64="${srcdir}/build-64" make } ``` -- https://gitlab.winehq.org/wine/wine/-/merge_requests/6715#note_88731
participants (5)
-
Alexandre Julliard (＠julliard) -
Chih-Hsuan Yen (＠yan12125)
-
Eric Pouech
-
eric pouech (＠epo)
-
Marvin