[Bug 60295] New: shell32: SHELL32_GetCustomFolderAttributeFromPath performs uncached desktop.ini CreateFile lookup at every ancestor directory on each shell attribute query, causing severe slowdown with large directory listings
http://bugs.winehq.org/show_bug.cgi?id=60295 Bug ID: 60295 Summary: shell32: SHELL32_GetCustomFolderAttributeFromPath performs uncached desktop.ini CreateFile lookup at every ancestor directory on each shell attribute query, causing severe slowdown with large directory listings Product: Wine Version: 11.0 Hardware: x86-64 OS: Linux Status: UNCONFIRMED Severity: critical Priority: P2 Component: shell32 Assignee: wine-bugs@list.winehq.org Reporter: kellnerp@earthlink.net Target Milestone: --- Distribution: --- Created attachment 82041 --> http://bugs.winehq.org/attachment.cgi?id=82041 Discussion of backtrace ShellItem_GetAttributes() in dlls/shell32/shellitem.c walks the full path from the shell namespace root to the target item via repeated SHELL32_BindToChild() calls (one per path component). At every level of this walk, SHELL32_GetCustomFolderAttributeFromPath() (dlls/shell32/shlfolder.c:75) performs a blocking GetPrivateProfileStringW() call against that directory's desktop.ini, which in turn does a full CreateFileW/NtCreateFile syscall via PROFILE_Open() (dlls/kernel32/profile.c:764). This lookup is not cached per-directory. Every call to ShellItem_GetAttributes() for an item N levels deep triggers N blocking file-open attempts, regardless of whether the directory has already been queried in this same operation. For workloads that call shell attribute queries once per file across a large, deeply-nested directory (e.g., an application enumerating thousands of files in a single folder and querying SFGAO_* attributes per item — this is what triggered the report, via Insta360 Studio, a Qt5-based app, importing a folder of several thousand sequentially-named interval/timelapse photos), this becomes O(files × path_depth) blocking file-open syscalls, each requiring a full wineserver round-trip. With a large enough file count this manifests as the application appearing to hang or freeze for an extended period, even though it is not deadlocked — it is making very slow forward progress. Reproduction Create a moderately deep directory path (~8-10 levels) on any filesystem. Place several thousand files in the deepest directory. Run any application that queries IShellItem::GetAttributes (or equivalent, e.g. via IShellFolder::BindToObject/GetAttributesOf) once per file while enumerating that directory — a Qt5 QFileDialog-based file picker with custom icon/grouping logic reliably triggers it. Observe severe slowdown proportional to file count × path depth, compared to native Windows behavior with the same file count. System details in attachement. Further testing will be done shortly hopefully with the development version. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60295 --- Comment #1 from TOP <kellnerp@earthlink.net> --- Comment for https://bugs.winehq.org/show_bug.cgi?id=39773 --- Ran into this same issue (uncached `desktop.ini` lookup in `SHELL32_GetCustomFolderAttributeFromPath()`) via a different real-world trigger than the original report, and did a fairly deep investigation that I think adds useful evidence. Posting a summary; happy to attach the reproduction source files if useful. **Real-world trigger:** Insta360 Studio 2022 (Qt5-based app running under Wine), when shift-click range-selecting a large sequence of files (~1000 interval/timelapse photos) in its file browser. No network share needed to reproduce the severity — a local filesystem with a moderately deep path (~10 components) and a large selection is enough to turn this into a multi-minute apparent hang requiring force-quit. **Confirmed mechanism (two independent live backtraces via `winedbg --gdb` on the hung process):** Main UI thread found blocked in `NtCreateFile`, with the following call chain (frame numbers as captured; some `SHELL32_BindToChild` recursion frames omitted for brevity): ``` NtCreateFile -> CreateFileW dlls/kernelbase/file.c:873 -> PROFILE_Open dlls/kernel32/profile.c:764 -> GetPrivateProfileStringW dlls/kernel32/profile.c:1422 -> SHELL32_GetCustomFolderAttributeFromPath dlls/shell32/shlfolder.c:75 -> SHELL32_BindToChild (recursing, once per path component) -> ShellItem_get_parent_shellfolder dlls/shell32/shellitem.c:165 -> ShellItem_GetAttributes dlls/shell32/shellitem.c:285 -> [caller in platforms/qwindows.dll -- Qt5 platform plugin] ``` Confirmed twice, independently, on different sessions/paths -- same mechanism both times. **Source-level confirmation of why this compounds so badly:** in `dlls/shell32/shlview.c`, `LV_AddItem()` inserts list items with: ```c lvItem.pszText = LPSTR_TEXTCALLBACKW; /* get text on a callback basis */ lvItem.iImage = I_IMAGECALLBACK; /* get the image on a callback basis */ ``` i.e. the ListView is populated in owner-data/callback mode. It does not hold display data -- it re-requests it via `LVN_GETDISPINFO` every time it needs to *show* an item (initial paint, scroll, selection-state change, any redraw). Each of those callbacks resolves through the shell folder, hitting the uncached `desktop.ini` walk again. This means the real-world cost isn't "N files x path-depth" as a single pass -- it's that multiplied again by however many times the UI redraws each visible item during the operation. **Quantitative confirmation**, using a 1000-file/10-level-deep synthetic test tree (fully synthetic, reproducible without Insta360): | Test | Per-item cost | Notes | |---|---|---| | Headless console app, single `IShellItem::GetAttributes()` call per file, no UI | 2.1 ms | Baseline: raw uncached-lookup cost only | | Custom listview replicating shlview.c's exact `LPSTR_TEXTCALLBACKW` pattern, `strace`'d | ~399 `openat` syscalls/file | Manual real interaction (shift-click), not automated | | Real `GetOpenFileNameW()` dialog (Wine's actual comdlg32/shell32), manual shift-click | ~76 ms/item | Two independently-built repro programs converge on ~76-77ms/item and similar syscall-count order of magnitude | | Real Insta360 Studio, live process, `strace`'d during actual hang | ~534 `openat` syscalls/file, ~750ms/item wall time | 947-file real selection | The call-count order of magnitude (mid-hundreds of opens per file) is consistent across three independently-measured sources: the real app, a minimal custom owner-data listview, and the real Wine common dialog. The residual gap between synthetic repro wall-time (~76ms/item) and the real app (~750ms/item) is most likely wineserver round-trip contention from Studio's ~80 background threads competing for the same serialized socket during selection -- not reproducible in an isolated single-process test by design, and not needed to explain the core mechanism or its severity. **Confirms this comment's suggested fix (from the original report) still applies and is the right fix:** caching `SHELL32_GetCustomFolderAttributeFromPath()`'s result per directory (rather than returning FALSE unconditionally, which would break legitimate custom-folder-icon support) would collapse the repeated-lookup cost to one real check per unique directory for the lifetime of a shell operation, regardless of how many times the owner-data listview re-queries display info per item. This matches how native Windows Explorer behaves (per the original report: "when displaying a filename we have everything in the cache so redisplays are lightning fast"). Still reproducible on current Wine (tested on 11.0), a decade after the original report -- this remains an open, real-world-impacting issue. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60295 Ken Sharp <imwellcushtymelike@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|critical |trivial --- Comment #2 from Ken Sharp <imwellcushtymelike@gmail.com> --- Not remotely critical. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60295 Ken Sharp <imwellcushtymelike@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |performance -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60295 Ken Sharp <imwellcushtymelike@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |DUPLICATE Status|UNCONFIRMED |RESOLVED --- Comment #3 from Ken Sharp <imwellcushtymelike@gmail.com> --- And you already found the duplicate bug, so why open another one? *** This bug has been marked as a duplicate of bug 39773 *** -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60295 --- Comment #4 from TOP <kellnerp@earthlink.net> --- (In reply to Ken Sharp from comment #2)
Not remotely critical.
Makes Insta360 Studio unusable when working with interval photos. That is not trivial when it breaks software. Saying it is resolved when it very much isn't doesn't work for me. Bug 39773 is not resolved or this problem would not crop up. AFAIK the problematic code still exists in the development versions. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=60295 Zeb Figura <z.figura12@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |z.figura12@gmail.com Status|RESOLVED |CLOSED --- Comment #5 from Zeb Figura <z.figura12@gmail.com> --- (In reply to TOP from comment #4)
(In reply to Ken Sharp from comment #2)
Not remotely critical.
Makes Insta360 Studio unusable when working with interval photos. That is not trivial when it breaks software.
Our importance field has specific documented meanings; "blocker" means it blocks development, which this doesn't. Although, to be honest, I doubt anyone actually cares or even notices what they are for any given bug, so fussing over the importance of individual bugs is probably not a worthwhile endeavour, and we should probably just remove the field altogether if we can.
Saying it is resolved when it very much isn't doesn't work for me.
Bug 39773 is not resolved or this problem would not crop up. AFAIK the problematic code still exists in the development versions.
Bug 39773 isn't marked as resolved. What's the problem here? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
participants (1)
-
WineHQ Bugzilla