[Bug 53683] New: Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

Bug ID: 53683
           Summary: Mod Organzer 2 fails to read random directories and
                    files
           Product: Wine-staging
           Version: 7.17
          Hardware: x86-64
               URL: https://mega.nz/file/S8R2RbiL#YNLrxvQvb2xuneLqbw_TtOKm
                    v7aDQ2A7cLWw_FbYbFY
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: -unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
                CC: [e-mail], [e-mail],
                    [e-mail]
   Regression SHA1: 1ed196f0effa262b297705f36975261f44001417
      Distribution: ArchLinux

Created attachment 73094
  --> https://bugs.winehq.org/attachment.cgi?id=73094
TTY log

As of Wine-staging 1ed196f0effa262b297705f36975261f44001417 Mod Organizer will
fail to read random files and folders.

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #1 from Erich E. Hoover  ---
(In reply to gem-man-alt from comment #0)
> Created attachment 73094 [details]
> TTY log
>
> As of Wine-staging 1ed196f0effa262b297705f36975261f44001417 Mod Organizer
> will fail to read random files and folders.

Thanks for letting me know, do the files and folders that are not working
happen to be Unix symlinks? (trying to narrow in on what's going on)

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #2 from [e-mail] ---
They are not. Just regular files and folders, and there seams to be no
consistency in which ones it fails to read and they are different files and
folders every time the program is launched.

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #3 from Erich E. Hoover  ---
(In reply to gem-man-alt from comment #2)
> They are not. Just regular files and folders, and there seams to be no
> consistency in which ones it fails to read and they are different files and
> folders every time the program is launched.

Hmm, that seems strange.  If this application doesn't specifically do anything
with reparse points (most apps) then I would guess that it's a flaw in checking
whether something is a reparse point.  Try disabling the check for something
being a reparse point when parsing paths in patch 11:
https://github.com/wine-staging/wine-staging/blob/master/patches/ntdll-Junct...

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #4 from [e-mail] ---
(In reply to Erich E. Hoover from comment #3)
> (In reply to gem-man-alt from comment #2)
> > They are not. Just regular files and folders, and there seams to be no
> > consistency in which ones it fails to read and they are different files and
> > folders every time the program is launched.
>
> Hmm, that seems strange.  If this application doesn't specifically do
> anything with reparse points (most apps) then I would guess that it's a flaw
> in checking whether something is a reparse point.  Try disabling the check
> for something being a reparse point when parsing paths in patch 11:
> https://github.com/wine-staging/wine-staging/blob/master/patches/ntdll-
> Junction_Points/0011-ntdll-Follow-reparse-points-during-path-resolution.
> patch#L186

Sorry, exactly which lines do I comment to disable the check? I know nothing
about programming

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #5 from Erich E. Hoover  ---
(In reply to gem-man-alt from comment #4)
> ...
> Sorry, exactly which lines do I comment to disable the check? I know nothing
> about programming

This is probably the easiest thing to do (remove the entire new block):
===
#if 0
        if (find_reparse_target( unix_name, &target, &target_len ) ==
STATUS_SUCCESS)
...
        else if (!name_len)
#else
        if (!name_len)
#endif
===

Is there an easy way for me to test with this program?

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #6 from [e-mail] ---
> This is probably the easiest thing to do (remove the entire new block):
>
> #if 0
>         if (find_reparse_target( unix_name, &target, &target_len ) ==
> STATUS_SUCCESS)
> ...
>         else if (!name_len)
> #else
>         if (!name_len)
> #endif
>
> Is there an easy way for me to test with this program?

The simplest way to test the program if you own any Fallout or Elder Scrolls
games would be to install that game, install MO2 into the the same prefix, it
should autodetect the game, then install a simple mod (second button on the
toolbar, for Skyrim here is an empty esp
https://www.nexusmods.com/skyrimspecialedition/mods/23658), then you can make
copies of the esp with new names in the folder it installs it to until the bug
starts reliably triggering (the more files it is managing the more likely the
bug is to trigger and the more severe it gets). Mod organizer will tell you
about any plugins it fails to find

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #7 from [e-mail] ---
It's probably me being stupid but I can't get the patch to apply. Here are the
changes I made
https://github.com/Ember2528/wine-staging/blob/92d726cb3b903859ab2d1737beecc...

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #8 from Erich E. Hoover  ---
(In reply to gem-man-alt from comment #7)
> It's probably me being stupid but I can't get the patch to apply. Here are
> the changes I made
> https://github.com/Ember2528/wine-staging/blob/
> 92d726cb3b903859ab2d1737beeccb5502cf4832/patches/ntdll-Junction_Points/0011-
> ntdll-Follow-reparse-points-during-path-resolution.patch

If you're editing the patch (rather than editing the file afterwards) then just
remove this entire block:
===
@@ -3999,8 +4070,39 @@ static NTSTATUS lookup_unix_name( const WCHAR *name, int
name_len, char **buffer

status = find_file_in_dir( unix_name, pos, name, end - name, is_unix
);

#if 0
        if (find_reparse_target( unix_name, &target, &target_len ) ==
STATUS_SUCCESS)
...
        else if (!name_len)
#else
        if (!name_len)
#endif
===

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

Béla Gyebrószki  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #9 from Béla Gyebrószki  ---
@OP: Could you try Erich's proposed solution from bug #53719,comment #1 ?

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #10 from Erich E. Hoover  ---
Created attachment 73169
  --> https://bugs.winehq.org/attachment.cgi?id=73169
replacement for reparse point patch 11

(In reply to Béla Gyebrószki from comment #9)
> @OP: Could you try Erich's proposed solution from bug #53719,comment #1 ?

OP probably needs a replacement patch, @OP try this attachment.

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #11 from [e-mail] ---
(In reply to Erich E. Hoover from comment #10)
> Created attachment 73169 [details]
> replacement for reparse point patch 11
>
> (In reply to Béla Gyebrószki from comment #9)
> > @OP: Could you try Erich's proposed solution from bug #53719,comment #1 ?
>
> OP probably needs a replacement patch, @OP try this attachment.

The changes from bug #53719 don't seam to make any difference for this issue.

And as for removing the whole section as you suggested in comment #5 all that
does it make it so MO2 can't read anything at all and fails to load the
instance

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

Tuupertunut  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #12 from Tuupertunut  ---
My Arma 3 also fails to find random files and directories, and I think I found
the reason.

https://github.com/wine-staging/wine-staging/blob/08ad4107613a980f533f4784a2...

Here this patch is using chdir(), which also affects other threads in the same
process. Whenever this code runs, for a little while, all other threads see the
wrong working directory and therefore fail to read any files with relative
paths during this period. Later the original working directory is restored with
fchdir() and the files can be found again.

You should not use chdir() when traversing path elements but for example use
file descriptors instead.

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #13 from Erich E. Hoover  ---
(In reply to Tuupertunut from comment #12)
> ...
> You should not use chdir() when traversing path elements but for example use
> file descriptors instead.

I've been looking at simplifying this and this routine actually works with
absolute paths the whole time, so you should be able to remove all the chdir
nonsense.  If you are willing to test that for me then I would appreciate it.

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #14 from Tuupertunut  ---
Are you saying that just removing the lines that use chdir should be enough?

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #15 from [e-mail] ---
The issue appears to be fixed in in staging
5be23b8ae889d8b8f6e7f198d29807d926057d50

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

Zeb Figura  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|UNCONFIRMED                 |RESOLVED
      Fixed by SHA1|                            |769ddd9f009d8b209e840a8d32b
                   |                            |b05f43cbe4295

--- Comment #16 from Zeb Figura  ---
Fixed by wine-staging commit 769ddd9f009d8b209e840a8d32bb05f43cbe4295, thanks
for testing.

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

--- Comment #17 from Erich E. Hoover  ---
(In reply to Tuupertunut from comment #14)
> Are you saying that just removing the lines that use chdir should be enough?

Sorry, somehow I missed this message.  For most cases that would have been
enough (it would only break for certain reparse points), but the updated
patchset should fix this issue for you.  The new version handles everything as
full file paths (stitching together paths as necessary), so the chdir() is gone
now.

[Bug 53683] Mod Organzer 2 fails to read random directories and files

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=53683

Alistair Leslie-Hughes  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #18 from Alistair Leslie-Hughes  ---
Closing Fixed wine-staging bug.