[Bug 50586] New: [regression] NtQueryInformationFile returning STATUS_INVALID_INFO_CLASS (0xc0000003) on symlinks opened with option FILE_OPEN_REPARSE_POINT (0x00200000)
https://bugs.winehq.org/show_bug.cgi?id=50586 Bug ID: 50586 Summary: [regression] NtQueryInformationFile returning STATUS_INVALID_INFO_CLASS (0xc0000003) on symlinks opened with option FILE_OPEN_REPARSE_POINT (0x00200000) Product: Wine-staging Version: 6.0 Hardware: x86-64 OS: Linux Status: UNCONFIRMED Severity: normal Priority: P2 Component: -unknown Assignee: wine-bugs(a)winehq.org Reporter: mr.dovi(a)gmail.com CC: leslie_alistair(a)hotmail.com, z.figura12(a)gmail.com Distribution: --- Created attachment 69263 --> https://bugs.winehq.org/attachment.cgi?id=69263 wine_file.c testcase Possible related issue: https://bugs.winehq.org/show_bug.cgi?id=50036 I'm investigating the cause of a application startup crash happening on wine-staging 6.0 but not on wine-staging-5.21, wine-5.13 and wine-6.0 While I think I have found a workaround to have it running on staging-6.0, I realized it may be worth a share since it took me a few hours of searching the relay logs. I have attached a test case but basically, calling NtQueryInformationFile on a symlink opened with the file handle option FILE_OPEN_REPARSE_POINT seem to cause NtQueryInformationFile to return STATUS_INVALID_INFO_CLASS, even if the class is correctly set to FileAllInformation # wine 5.13 # pNtCreateFile -> 0 # NtQueryInformationFile -> 0 # wine 5.13 # pNtCreateFile -> 0 # NtQueryInformationFile -> 0 # wine-staging 5.21 # pNtCreateFile -> 0 # NtQueryInformationFile -> 0 # wine-staging 5.21 # pNtCreateFile -> 0 # NtQueryInformationFile -> 0 # wine-staging 6.0 # pNtCreateFile -> 0 # NtQueryInformationFile -> c0000003 # wine-staging 6.0 # pNtCreateFile -> 0 # NtQueryInformationFile -> 0 My system is Arch Linux -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #1 from Arnaud Dovi <mr.dovi(a)gmail.com> --- I had removed the commands from OP for readability but basically I have done 2 testcase per wine version, one on a symlink directory and another one on a real directory wine_file c:\symlink wine_file c:\windows To note that wine-staging-6.0 returns 0 with real directories so it seems to only affect symlinks. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 Zebediah Figura <z.figura12(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |erich.e.hoover(a)gmail.com --- Comment #2 from Zebediah Figura <z.figura12(a)gmail.com> --- Erich, can you please take a look at this? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #3 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Zebediah Figura from comment #2)
Erich, can you please take a look at this?
Absolutely, I can take a look tomorrow. I also have some other updates to the patches (from the other bugs), but I didn't want to potentially cause unintended issues for the 6.0 release. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 Gijs Vermeulen <gijsvrm(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|[regression] |NtQueryInformationFile |NtQueryInformationFile |returns |returning |STATUS_INVALID_INFO_CLASS |STATUS_INVALID_INFO_CLASS |on symlinks opened with |(0xc0000003) on symlinks |FILE_OPEN_REPARSE_POINT |opened with option | |FILE_OPEN_REPARSE_POINT | |(0x00200000) | Keywords| |regression -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #4 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Arnaud Dovi from comment #1)
I had removed the commands from OP for readability but basically I have done 2 testcase per wine version, one on a symlink directory and another one on a real directory
wine_file c:\symlink wine_file c:\windows
To note that wine-staging-6.0 returns 0 with real directories so it seems to only affect symlinks.
Okay, before I back out and test the old patches I have a question: what kind of symlink are you using? There are four types (at the moment): 1) junction point 2) unix symlink 3) relative directory symlink 4) absolute directory symlink examples: === c:\>dir Volume in drive c has no label. Volume Serial Number is 0000-0000 Directory of c:\ 1/30/2021 12:55 PM <JUNCTION> junction [C:\windows] 1/30/2021 12:55 PM <SYMLINKD> symlink 1/30/2021 12:55 PM <SYMLINKD> symlinkd [windows] 1/30/2021 12:55 PM <SYMLINKD> symlinkdabs [C:\windows] ... === With your program I see: * c:\junction: pNtCreateFile -> 0 NtQueryInformationFile -> 0 * c:\symlink: pNtCreateFile -> 0 NtQueryInformationFile -> c0000003 * c:\symlinkd: pNtCreateFile -> 0 NtQueryInformationFile -> 0 * c:\symlinkdabs: pNtCreateFile -> 0 NtQueryInformationFile -> 0 So, I suspect that this problem only extends to "unix symlinks" - which I have not decided how to treat "properly" yet. My current inclination is to convert them into Z:\ paths, but that may have weird consequences. If this is actually the problem for you, what was happening before? It's been a while since I made the updates for 6.0, but I think that before then it reported by-handle requests as the information for the destination folder (didn't show up as a symlink). -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #5 from Arnaud Dovi <mr.dovi(a)gmail.com> --- (In reply to Erich E. Hoover from comment #4)
(In reply to Arnaud Dovi from comment #1)
I had removed the commands from OP for readability but basically I have done 2 testcase per wine version, one on a symlink directory and another one on a real directory
wine_file c:\symlink wine_file c:\windows
To note that wine-staging-6.0 returns 0 with real directories so it seems to only affect symlinks.
Okay, before I back out and test the old patches I have a question: what kind of symlink are you using? There are four types (at the moment): 1) junction point 2) unix symlink 3) relative directory symlink 4) absolute directory symlink
Wasn't aware there was 4 different one, the one I used is what I think is 2) unix symlink, created with ln -s on a directory. I also know the MKLINK.exe method but this one does not ships with wine so they are probably only creatable programmatically.
examples: === c:\>dir Volume in drive c has no label. Volume Serial Number is 0000-0000
Directory of c:\
1/30/2021 12:55 PM <JUNCTION> junction [C:\windows] 1/30/2021 12:55 PM <SYMLINKD> symlink 1/30/2021 12:55 PM <SYMLINKD> symlinkd [windows] 1/30/2021 12:55 PM <SYMLINKD> symlinkdabs [C:\windows] ... ===
With your program I see: * c:\junction: pNtCreateFile -> 0 NtQueryInformationFile -> 0 * c:\symlink: pNtCreateFile -> 0 NtQueryInformationFile -> c0000003 * c:\symlinkd: pNtCreateFile -> 0 NtQueryInformationFile -> 0 * c:\symlinkdabs: pNtCreateFile -> 0 NtQueryInformationFile -> 0
So, I suspect that this problem only extends to "unix symlinks" - which I have not decided how to treat "properly" yet. My current inclination is to convert them into Z:\ paths, but that may have weird consequences. If this is actually the problem for you, what was happening before? It's been a while since I made the updates for 6.0, but I think that before then it reported by-handle requests as the information for the destination folder (didn't show up as a symlink).
I think in the startup log OK I have the link is converted to Z On logs side by side the break happens here on staging-6.0 trace:file:nt_to_unix_file_name_internal L"\\??\\C:\\Program Files\\Black Tree Gaming Ltd\\Vortex" -> "/home/arno/.local/share/Steam/steamapps/compatdata/2260996941/pfx/dosdevices/c:/Program Files/Black Tree Gaming Ltd/Vortex" create_file( access=00100080, sharing=00000007, create=1, options=00204020, attrs=00000000, objattr={rootdir=0000,attributes=00000040,sd={},name=L""}, filename="/home/arno/.local/share/Steam/steamapps/compatdata/2260996941/pfx/dosdevices/c:/Program Files/Black Tree Gaming Ltd/Vortex" ) create_file() = 0 { handle=0240 } Ret ntdll.NtCreateFile() retval=00000000 ret=7b016bea Call ntdll.RtlFreeUnicodeString(0021d180) ret=7b016fdd Ret ntdll.RtlFreeUnicodeString() retval=00000001 ret=7b016fdd trace:file:CreateFileW returning 0000000000000240 Ret KERNEL32.CreateFileW() retval=00000240 ret=142ee27b1 Call ntdll.NtQueryInformationFile(00000240,0021d360,0021d388,00000068,00000012) ret=142ee6ddb trace:file:NtQueryInformationFile (0x240,0x21d360,0x21d388,0x00000068,0x00000012) get_handle_fd( handle=0240 ) *fd* 0240 -> 296 get_handle_fd() = 0 { type=2, cacheable=1, access=00100080, options=00204020 } Ret ntdll.NtQueryInformationFile() retval=c0000003 ret=142ee6ddb Call KERNEL32.GetLastError() ret=142ee27d9 Ret KERNEL32.GetLastError() retval=00000057 ret=142ee27d9 while one staging-5.21 this seem to resolve to Z: trace:file:nt_to_unix_file_name_internal L"\\??\\C:\\Program Files\\Black Tree Gaming Ltd\\Vortex" -> "/home/arno/.local/share/Steam/steamapps/compatdata/2260996941/pfx/dosdevices/c:/Program Files/Black Tree Gaming Ltd/Vortex" create_file( access=00100080, sharing=00000007, create=1, options=00004020, attrs=00000000, objattr={rootdir=0000,attributes=00000040,sd={},name=L""}, filename="/home/arno/.local/share/Steam/steamapps/compatdata/2260996941/pfx/dosdevices/c:/Program Files/Black Tree Gaming Ltd/Vortex" ) create_file() = 0 { handle=0218 } Ret ntdll.NtCreateFile() retval=00000000 ret=7b016a31 Call ntdll.RtlFreeUnicodeString(0021d1d0) ret=7b016ed5 Ret ntdll.RtlFreeUnicodeString() retval=00000001 ret=7b016ed5 trace:file:CreateFileW returning 0000000000000218 Ret KERNEL32.CreateFileW() retval=00000218 ret=142ee27b1 Call ntdll.NtQueryInformationFile(00000218,0021d360,0021d388,00000068,00000012) ret=142ee6ddb trace:file:NtQueryInformationFile (0x218,0x21d360,0x21d388,0x00000068,0x00000012) get_handle_fd( handle=0218 ) *fd* 0218 -> 324 get_handle_fd() = 0 { type=2, cacheable=1, access=00100080, options=00004020 } get_handle_unix_name( handle=0218, nofollow=0 ) get_handle_unix_name() = 0 { name_len=65, name="/home/arno/.local/share/Steam/steamapps/common/Vortex Mod Manager" } trace:file:find_drive_rootA "/home/arno/.local/share/Steam/steamapps/common/Vortex Mod Manager" -> drive Z:, root="/", name="/home/arno/.local/share/Steam/steamapps/common/Vortex Mod Manager" To note that in some part of the staging-6.0 failing log, NtQueryInformationFile returns OK and the link is resolved if the options in create_file is 00004020 not 00204020 (FILE_OPEN_REPARSE_POINT removed) -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #6 from Arnaud Dovi <mr.dovi(a)gmail.com> --- Created attachment 69282 --> https://bugs.winehq.org/attachment.cgi?id=69282 startup logs -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #7 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Arnaud Dovi from comment #5)
... Wasn't aware there was 4 different one, the one I used is what I think is 2) unix symlink, created with ln -s on a directory. I also know the MKLINK.exe method but this one does not ships with wine so they are probably only creatable programmatically.
Not a problem, I was just worried that I had accidentally broken something that a Windows program was doing (but they cannot create unix symlinks). wine-staging does support mklink, but it's currently treated as a builtin command in Wine's cmd. For example (absolute directory symlink): wine cmd /C 'mklink /d C:\\target C:\\windows'
... To note that in some part of the staging-6.0 failing log, NtQueryInformationFile returns OK and the link is resolved if the options in create_file is 00004020 not 00204020 (FILE_OPEN_REPARSE_POINT removed)
Okay, it looks like what would have happened before (FILE_OPEN_REPARSE_POINT did not work correctly) was that pre-6.0 wine-staging would have followed the link to the target and reported information on the target. So, I have a few sane ways that I can handle this that I'm considering: 1) treat unix symlinks as an NT symlink (0xA000000C) targeting the Z:\ or \??\unix path 2) treat unix symlinks as a custom NT reparse point ('UNIX'/0x58494E55?) targeting the unmodified unix path 3) treat unix symlinks as not being NT reparse points at all and always follow them to the target I'm leaning towards option 2 to try and keep unix symlinks distinct from the NT symlinks and to potentially allow programs built to work with wine to create unix symlinks if they wish. The downside of this option is that it's possible that poorly-coded Windows programs would see the unusual reparse tag and do something dumb. I'm open to other opinions on the matter (option 1 is easiest to implement, BTW), so if you have thoughts I'd be interested to hear them. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #8 from Hans Leidekker <hans(a)meelstraat.net> --- (In reply to Erich E. Hoover from comment #7)
Okay, it looks like what would have happened before (FILE_OPEN_REPARSE_POINT did not work correctly) was that pre-6.0 wine-staging would have followed the link to the target and reported information on the target. So, I have a few sane ways that I can handle this that I'm considering: 1) treat unix symlinks as an NT symlink (0xA000000C) targeting the Z:\ or \??\unix path 2) treat unix symlinks as a custom NT reparse point ('UNIX'/0x58494E55?) targeting the unmodified unix path 3) treat unix symlinks as not being NT reparse points at all and always follow them to the target
I'm leaning towards option 2 to try and keep unix symlinks distinct from the NT symlinks and to potentially allow programs built to work with wine to create unix symlinks if they wish. The downside of this option is that it's possible that poorly-coded Windows programs would see the unusual reparse tag and do something dumb.
Doesn't this argue for 3? If an app is Wine-aware there should be other ways to create unix symlinks. I think transparent unix symlinks is a useful feature that users may already depend on. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #9 from Zebediah Figura <z.figura12(a)gmail.com> --- Note that a symlink can span file systems. It seems possible that a Windows application would be confused about a file residing on a different file system than its containing directory. That's already the case, and maybe not worth worrying about, but if we're going to make a grand final design of how symlinks work it might be worth taking into account... -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #10 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Hans Leidekker from comment #8)
(In reply to Erich E. Hoover from comment #7)
... I'm leaning towards option 2 to try and keep unix symlinks distinct from the NT symlinks and to potentially allow programs built to work with wine to create unix symlinks if they wish. The downside of this option is that it's possible that poorly-coded Windows programs would see the unusual reparse tag and do something dumb.
Doesn't this argue for 3? If an app is Wine-aware there should be other ways to create unix symlinks. I think transparent unix symlinks is a useful feature that users may already depend on.
I've actually put together a little tech demo on completely transparent symlinks to reduce the size of the prefix by using symlinks for the fake/real dlls. It would not be too difficult for me to modify this to look for "no recognized tag" instead of "special Wine tag". I can probably do that this weekend if you think that's worth it. (In reply to Zebediah Figura from comment #9)
Note that a symlink can span file systems. It seems possible that a Windows application would be confused about a file residing on a different file system than its containing directory. That's already the case, and maybe not worth worrying about, but if we're going to make a grand final design of how symlinks work it might be worth taking into account...
Are you talking about applications that look for files based on inode? We could conceivably report the inode of the symlink in the case of a transparent symlink like this (and this is a feature that wouldn't be too hard to add in the future). -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #11 from Zebediah Figura <z.figura12(a)gmail.com> --- (In reply to Erich E. Hoover from comment #10)
(In reply to Zebediah Figura from comment #9)
Note that a symlink can span file systems. It seems possible that a Windows application would be confused about a file residing on a different file system than its containing directory. That's already the case, and maybe not worth worrying about, but if we're going to make a grand final design of how symlinks work it might be worth taking into account...
Are you talking about applications that look for files based on inode? We could conceivably report the inode of the symlink in the case of a transparent symlink like this (and this is a feature that wouldn't be too hard to add in the future).
I'm not immediately sure it's the only problem, but it does come to mind as a concern. Note that the above solution won't work for directory symlinks. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #12 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Zebediah Figura from comment #11)
(In reply to Erich E. Hoover from comment #10)
... Are you talking about applications that look for files based on inode? We could conceivably report the inode of the symlink in the case of a transparent symlink like this (and this is a feature that wouldn't be too hard to add in the future).
I'm not immediately sure it's the only problem, but it does come to mind as a concern. Note that the above solution won't work for directory symlinks.
Yeah, I'm not sure what we could possibly do to properly support a cross-device directory unix symlink if someone's searching the disk by inode. If we treated them as NT symlinks then any application doing this is _supposed_ to use GetVolumePathName to search the correct volume. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #13 from Zebediah Figura <z.figura12(a)gmail.com> --- (In reply to Erich E. Hoover from comment #12)
Yeah, I'm not sure what we could possibly do to properly support a cross-device directory unix symlink if someone's searching the disk by inode. If we treated them as NT symlinks then any application doing this is _supposed_ to use GetVolumePathName to search the correct volume.
Right, and that actually works. It's not clear to me if there are any problems that result from this (well, I've seen a couple applications not know what a Windows mounted folder is, but that can probably be fixed by always automatically creating a drive for any mount point). But I'm probably missing something. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 soredake <gi85qht0z(a)relay.firefox.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |gi85qht0z(a)relay.firefox.com -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #14 from Hans Leidekker <hans(a)meelstraat.net> --- (In reply to Zebediah Figura from comment #13)
(In reply to Erich E. Hoover from comment #12)
Yeah, I'm not sure what we could possibly do to properly support a cross-device directory unix symlink if someone's searching the disk by inode. If we treated them as NT symlinks then any application doing this is _supposed_ to use GetVolumePathName to search the correct volume.
Right, and that actually works. It's not clear to me if there are any problems that result from this (well, I've seen a couple applications not know what a Windows mounted folder is, but that can probably be fixed by always automatically creating a drive for any mount point). But I'm probably missing something.
If cross-device *directory* unix symlinks where NT symlinks but not cross-device *file* unix symlinks then I think you could end up with inconsistencies. E.g. GetVolumePathName would report different volumes for a file and its parent directory if the file is a symlink to a third filesystem. I guess you are referring to file ids being 128-bit and directory ids 64-bit? So we could include st_dev in file ids to make them unique across filesystems, but there's no such option for directory ids. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #15 from Zebediah Figura <z.figura12(a)gmail.com> --- (In reply to Hans Leidekker from comment #14)
(In reply to Zebediah Figura from comment #13)
(In reply to Erich E. Hoover from comment #12)
Yeah, I'm not sure what we could possibly do to properly support a cross-device directory unix symlink if someone's searching the disk by inode. If we treated them as NT symlinks then any application doing this is _supposed_ to use GetVolumePathName to search the correct volume.
Right, and that actually works. It's not clear to me if there are any problems that result from this (well, I've seen a couple applications not know what a Windows mounted folder is, but that can probably be fixed by always automatically creating a drive for any mount point). But I'm probably missing something.
If cross-device *directory* unix symlinks where NT symlinks but not cross-device *file* unix symlinks then I think you could end up with inconsistencies. E.g. GetVolumePathName would report different volumes for a file and its parent directory if the file is a symlink to a third filesystem.
I'm not sure I understand. If we pretend that cross-device file symlinks aren't symlinks, then a file will always report the same device as its containing directory, won't it?
I guess you are referring to file ids being 128-bit and directory ids 64-bit? So we could include st_dev in file ids to make them unique across filesystems, but there's no such option for directory ids.
I don't think that's enough, though, is it? FileInternalInformation is only 64 bits. Or is it okay for that to not be unique? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #16 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Zebediah Figura from comment #15)
(In reply to Hans Leidekker from comment #14)
... If cross-device *directory* unix symlinks where NT symlinks but not cross-device *file* unix symlinks then I think you could end up with inconsistencies. E.g. GetVolumePathName would report different volumes for a file and its parent directory if the file is a symlink to a third filesystem.
I'm not sure I understand. If we pretend that cross-device file symlinks aren't symlinks, then a file will always report the same device as its containing directory, won't it? ...
If I'm understanding correctly, we're talking about (in the context of unix symlinks): 1) reporting file symlinks transparently (no evidence to the Windows program that they exist) 2) reporting file symlinks using the inode number of the symlink rather than the target so that inode searches work correctly 3) reporting directory symlinks as an NT directory symlink (0xA000000C) So, due to #2 combined with #3, GetVolumePathName will always see that a file symlink is on the same device as the parent directory and if that directory is on a different device than its parent then it may report that directory as the start of the volume. Example: C:\myfolder -> D:\myfolder D:\myfolder\myfile -> E:\myfile GetVolumePathName(C:\myfolder\myfile) would give C:\myfolder, where C:\myfolder reports the target inode on D:\ and C:\myfolder\myfile reports the symlink inode on D:\. If i'm understanding correctly then this sounds reasonable to me, if a little complicated to implement. My inclination would be start by making unix symlinks act as similar as possible to how they do in non-staging now (transparently report all information about the target, regardless of being for a file or a directory) and then add this complexity on from there once I'm sure nothing is broken. How does that sound to everyone? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #17 from Arnaud Dovi <mr.dovi(a)gmail.com> --- (In reply to Erich E. Hoover from comment #7)
(In reply to Arnaud Dovi from comment #5)
... Wasn't aware there was 4 different one, the one I used is what I think is 2) unix symlink, created with ln -s on a directory. I also know the MKLINK.exe method but this one does not ships with wine so they are probably only creatable programmatically.
Not a problem, I was just worried that I had accidentally broken something that a Windows program was doing (but they cannot create unix symlinks). wine-staging does support mklink, but it's currently treated as a builtin command in Wine's cmd. For example (absolute directory symlink): wine cmd /C 'mklink /d C:\\target C:\\windows'
Very nice, exactly what I was looking for the other day but I have only checked binaries available into system32 I will use this method instead of unix symlinks currently, thank you for sharing. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #18 from Zebediah Figura <z.figura12(a)gmail.com> --- (In reply to Erich E. Hoover from comment #16)
If I'm understanding correctly, we're talking about (in the context of unix symlinks): 1) reporting file symlinks transparently (no evidence to the Windows program that they exist) 2) reporting file symlinks using the inode number of the symlink rather than the target so that inode searches work correctly 3) reporting directory symlinks as an NT directory symlink (0xA000000C)
So, due to #2 combined with #3, GetVolumePathName will always see that a file symlink is on the same device as the parent directory and if that directory is on a different device than its parent then it may report that directory as the start of the volume. Example: C:\myfolder -> D:\myfolder D:\myfolder\myfile -> E:\myfile GetVolumePathName(C:\myfolder\myfile) would give C:\myfolder, where C:\myfolder reports the target inode on D:\ and C:\myfolder\myfile reports the symlink inode on D:\.
If i'm understanding correctly then this sounds reasonable to me, if a little complicated to implement. My inclination would be start by making unix symlinks act as similar as possible to how they do in non-staging now (transparently report all information about the target, regardless of being for a file or a directory) and then add this complexity on from there once I'm sure nothing is broken. How does that sound to everyone?
I think so, yeah. But on the other hand, two questions: (1) How much of a problem is it really to report Unix symlinks as win32 symlinks? (Does the wine-staging patch already do this?) It could potentially be simpler. Note also that we need to treat system DLLs specially anyway, to get copy-on-write behaviour. (2) On the other hand, if there is a good reason to hide Unix symlinks, does it make sense to also hide same-device Unix directory symlinks? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #19 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Zebediah Figura from comment #18)
...
(1) How much of a problem is it really to report Unix symlinks as win32 symlinks? (Does the wine-staging patch already do this?) It could potentially be simpler.
Not much, it's really just modifying the current patch to detect that the symlink is not Wine-created and convert the Unix path into an NT path. The concern about doing this would be for old applications, that don't have reparse point support, accessing unix symlinks. This isn't just user-created unix symlinks though, Wine creates unix symlinks for several special directories (Desktop, My Documents, etc.), so this matters even in a fresh prefix.
Note also that we need to treat system DLLs specially anyway, to get copy-on-write behaviour.
Right. While I haven't implemented that in the tech demo I gave you, that's part of why it uses a custom reparse tag. You almost certainly wouldn't want to do that with regular unix symlinks though.
(2) On the other hand, if there is a good reason to hide Unix symlinks, does it make sense to also hide same-device Unix directory symlinks?
Conceivably we could hide all same-device symlinks and report cross-device unix symlinks as NT symlinks (whether they're a directory or not). But we may be spending a lot of effort thinking about something that's not really a big deal, do you know if we have any bugs reported for issues like this? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #20 from Zebediah Figura <z.figura12(a)gmail.com> --- (In reply to Erich E. Hoover from comment #19)
(In reply to Zebediah Figura from comment #18)
...
(1) How much of a problem is it really to report Unix symlinks as win32 symlinks? (Does the wine-staging patch already do this?) It could potentially be simpler.
Not much, it's really just modifying the current patch to detect that the symlink is not Wine-created and convert the Unix path into an NT path. The concern about doing this would be for old applications, that don't have reparse point support, accessing unix symlinks. This isn't just user-created unix symlinks though, Wine creates unix symlinks for several special directories (Desktop, My Documents, etc.), so this matters even in a fresh prefix.
I guess I should clarify that I'm thinking about file symlinks here; I already assumed we'd want to always report directory symlinks as NT symlinks per the discussion above. (Note also that we've already reported directory symlinks as FILE_ATTRIBUTE_REPARSE_POINT since forever, though we haven't implemented really any of the other behaviour surrounding that.)
(2) On the other hand, if there is a good reason to hide Unix symlinks, does it make sense to also hide same-device Unix directory symlinks?
Conceivably we could hide all same-device symlinks and report cross-device unix symlinks as NT symlinks (whether they're a directory or not). But we may be spending a lot of effort thinking about something that's not really a big deal, do you know if we have any bugs reported for issues like this?
That's the real question, because honestly I don't, but I'm not sure I trust that. I have seen two different applications choke on mount points (bug 49840, bug 50557). I think (though I'm not sure) that both would be fixed by automatically assigning drive letters to all file systems (well, probably skipping procfs/devfs/sysfs/etc). I don't think I've seen any application choke on symlinks, however, cross-device or same-device, except for the occasional fixable bug in the wine-staging patches. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #21 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Zebediah Figura from comment #20)
(In reply to Erich E. Hoover from comment #19)
... (Note also that we've already reported directory symlinks as FILE_ATTRIBUTE_REPARSE_POINT since forever, though we haven't implemented really any of the other behaviour surrounding that.)
Right, so I would say my first priority should be to fix this bug by making the unix symlinks fully transparent again (ignore FILE_OPEN_REPARSE_POINT) since that's been our baseline for some time. If we run into some applications that actually care then it would not be too much trouble to change and report them "properly", so we could always re-evaluate at that point.
... But we may be spending a lot of effort thinking about something that's not really a big deal, do you know if we have any bugs reported for issues like this?
That's the real question, because honestly I don't, but I'm not sure I trust that.
Yeah, I understand that feeling. I think that we've found over the years of updating these patches that reparse tag support is far more complicated than otherwise would be expected. However, I think that each time that we've opened a new can of worms that we've made a lot of progress. FILE_OPEN_REPARSE_POINT may just be our most recent can, but I'm personally feeling like we're finally coming to a place where we have all of our bases covered.
... I don't think I've seen any application choke on symlinks, however, cross-device or same-device, except for the occasional fixable bug in the wine-staging patches.
That's good to hear, it definitely feels like a fair number of the bugs have been me doing something dumb rather than larger architectural issues. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #22 from Zebediah Figura <z.figura12(a)gmail.com> --- (In reply to Erich E. Hoover from comment #21)
(In reply to Zebediah Figura from comment #20)
(In reply to Erich E. Hoover from comment #19)
... (Note also that we've already reported directory symlinks as FILE_ATTRIBUTE_REPARSE_POINT since forever, though we haven't implemented really any of the other behaviour surrounding that.)
Right, so I would say my first priority should be to fix this bug by making the unix symlinks fully transparent again (ignore FILE_OPEN_REPARSE_POINT) since that's been our baseline for some time. If we run into some applications that actually care then it would not be too much trouble to change and report them "properly", so we could always re-evaluate at that point.
Eh, so I'm kind of trying to ask if this is the right solution. I.e. can we fix this by translating unix symlinks to regular NT symlinks always, and implementing FILE_OPEN_REPARSE_POINT appropriately, and will that have negative consequences? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 Rémi Bernon <rbernon(a)codeweavers.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rbernon(a)codeweavers.com --- Comment #23 from Rémi Bernon <rbernon(a)codeweavers.com> --- I only have a limited understanding of what information games would see, but for applications that choke on symlinks, I can mention CoD games anti-tamper (for instance CoD: WWII) that I know are broken with the way Wine currently reports them. IIRC it specifically checks ntdll.dll path, and steam_api64.dll that is expected to be located alongside the game executable, and breaks in both case if a symlink is involved somewhere that makes realpath report a different path. I haven't tested with the staging patch series, so I don't know precisely if the game can be made happy with more accurate reporting, but that could probably be a good test case. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #24 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Zebediah Figura from comment #22)
... Eh, so I'm kind of trying to ask if this is the right solution. I.e. can we fix this by translating unix symlinks to regular NT symlinks always, and implementing FILE_OPEN_REPARSE_POINT appropriately, and will that have negative consequences?
I don't know, but I'd be happy to tweak things this weekend and we can find out. (In reply to Rémi Bernon from comment #23)
... IIRC it specifically checks ntdll.dll path, and steam_api64.dll that is expected to be located alongside the game executable, and breaks in both case if a symlink is involved somewhere that makes realpath report a different path.
I haven't tested with the staging patch series, so I don't know precisely if the game can be made happy with more accurate reporting, but that could probably be a good test case.
I doubt that reporting the symlinks properly would make it happy if it is breaking with what non-staging Wine is currently doing. It might work with the tech demo I put together for custom "fully transparent" symlinks though, since the whole point of that is to make the system dlls into symlinks without upsetting anything. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #25 from Arnaud Dovi <mr.dovi(a)gmail.com> --- Sharing with you my user experience, I hope it will help you. First let's describe the application I'm testing on, it is called Vortex, it is a mod manager written to install mods for various games. I noted some difference in the behavior of the application with wine vs staging with the setup below c:\symlink\app.exe with c:\symlink linking to external dosdevice x:\appdir\ on wine (expected behavior) The app never knows about x:\appdir\ path at all. Everything belongs to c:\symlink, perfect on staging The base app.exe uses c:\symlink and never knows about x:\appdir\ perfect but part responsible of loading plugins, that is nodejs based, somehow discovers the path x:\appdir\ resulting in some modules being not found. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #26 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Arnaud Dovi from comment #25)
... c:\symlink\app.exe with c:\symlink linking to external dosdevice x:\appdir\ ...
Thanks Arnaud, would you mind recreating that symlink with the wine-staging mklink? It should be something like this: wine cmd /C 'mklink /d x:\\appdir c:\\symlink' I'm hoping that a symlink created this way (that properly reports itself as an "NT symlink") will work correctly. If that's the case then reporting unix symlinks as NT symlinks should fix your problem. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #27 from Arnaud Dovi <mr.dovi(a)gmail.com> --- (In reply to Erich E. Hoover from comment #26)
(In reply to Arnaud Dovi from comment #25)
... c:\symlink\app.exe with c:\symlink linking to external dosdevice x:\appdir\ ...
Thanks Arnaud, would you mind recreating that symlink with the wine-staging mklink? It should be something like this: wine cmd /C 'mklink /d x:\\appdir c:\\symlink' I'm hoping that a symlink created this way (that properly reports itself as an "NT symlink") will work correctly. If that's the case then reporting unix symlinks as NT symlinks should fix your problem.
Sorry I have not been enough clear, but when I said "on staging" this was MKLINK yes because the LN way lead to startup crash on staging with the NtQueryInformationFile response. The app continue to startup fine with MKLINK but then on use I have weird behaviour where the base app correctly starts under c:\symlink and loads dll under c:\symlink aswell , but the nodejs/electron base part of the app seem to load dll module .node files under the path x:\\appdir, this leads to half of the app being loaded correctly because some nodejs modules are not found. One thing I'm unsure yet is how a vanilla wine reacts to this situation with symlink made with MKLINK, maybe it is an issue upstream on wine ? Anyway don't worry about helping me fixing the issue, I think I have found a better way to match how Steam proton actually works. This time I mount a dosdevice x: symlinked to the Steam root, and I launch the application with "X:\steamapps\common\Vortex Mod Manager\Vortex.exe" and I sed-replace the new path in the system.reg, rather than starting it with "C:\Program Files\...\Vortex.exe" That way with minimal symlinks, everything is found under X:, the manager and games, and every binaries is maintained outside the proton prefix But if this issue lies on staging I hope a good test case for you to fix this issue that should be difficult to work on. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #28 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Arnaud Dovi from comment #27)
... One thing I'm unsure yet is how a vanilla wine reacts to this situation with symlink made with MKLINK, maybe it is an issue upstream on wine ?
The staging mklink creates a "specially-formatted" unix symlink that upstream wine just treats like any regular unix symlink. You can see this if you do a listing on the file vs. running realpath: $ ls -haltr ~/.wine/drive_c/symlink lrwxrwxrwx 1 ehoover ehoover 80 Feb 5 11:02 /home/ehoover/.wine/drive_c/symlink -> ///././/////////////////////////.//././/home/ehoover/.wine/dosdevices/c:/windows $ realpath ~/.wine/drive_c/symlink /home/ehoover/.wine/drive_c/windows
Anyway don't worry about helping me fixing the issue, I think I have found a better way to match how Steam proton actually works. ... But if this issue lies on staging I hope a good test case for you to fix this issue that should be difficult to work on.
Yeah, I am concerned that there is a deficiency of some kind in my NT symlink support if this is causing issues for you. I would expect that an application that is aware of NT symlinks (which this appears to be) would behave correctly in the described scenario. Would you mind running the application with WINEDEBUG="+volume" to see if it is calling GetVolumePathName and doing something strange with the results? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #29 from Arnaud Dovi <mr.dovi(a)gmail.com> --- Created attachment 69322 --> https://bugs.winehq.org/attachment.cgi?id=69322 steam-GetVolumePathName.log GetVolumePathName is not called here. Attaching the log file steam-16615000947893993472-GetVolumePathName.log I did WINEDEBUG="+loaddll,+volume" so you can see at one point it starts using the X: path. Could attach another log with +relay if can help you are welcome -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #30 from Arnaud Dovi <mr.dovi(a)gmail.com> --- (In reply to Erich E. Hoover from comment #28)
Yeah, I am concerned that there is a deficiency of some kind in my NT symlink support if this is causing issues for you. I would expect that an application that is aware of NT symlinks (which this appears to be) would behave correctly in the described scenario. Would you mind running the application with WINEDEBUG="+volume" to see if it is calling GetVolumePathName and doing something strange with the results?
I have attached the log but no call to GetVolumePathName is found -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #31 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Arnaud Dovi from comment #30)
... I have attached the log but no call to GetVolumePathName is found
Yeah, I'm trying to think of other ways to track this down. Do you know how to compile wine-staging yourself? It might be helpful to edit dlls/ntdll/unix/file.c:server_get_unix_name by changing this line: req->nofollow = nofollow; to: req->nofollow = TRUE; and seeing if it "fixes" the issue. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #32 from Arnaud Dovi <mr.dovi(a)gmail.com> --- (In reply to Erich E. Hoover from comment #31)
(In reply to Arnaud Dovi from comment #30)
... I have attached the log but no call to GetVolumePathName is found
Yeah, I'm trying to think of other ways to track this down. Do you know how to compile wine-staging yourself?
I use Arch btw ;) So yes this is easy with https://aur.archlinux.org/packages/wine-staging-git
It might be helpful to edit dlls/ntdll/unix/file.c:server_get_unix_name by changing this line: req->nofollow = nofollow; to: req->nofollow = TRUE; and seeing if it "fixes" the issue.
I did this change but unfortunately same result -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #33 from Arnaud Dovi <mr.dovi(a)gmail.com> --- Created attachment 69353 --> https://bugs.winehq.org/attachment.cgi?id=69353 steps to reproduce Hope it helps -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #34 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- Created attachment 69360 --> https://bugs.winehq.org/attachment.cgi?id=69360 report unrecognized tag (In reply to Arnaud Dovi from comment #33)
Created attachment 69353 [details] steps to reproduce
Hope it helps
That reproduces the error on startup really effectively, thanks. However, I don't know how to tell if I've fixed the (In reply to Arnaud Dovi from comment #33)
Created attachment 69353 [details] steps to reproduce
Hope it helps
Yes, thank you. I removed the internal uses where we get the link target, so I can confirm that this application is aware of reparse tags and that it is effectively responsible for the "problem": === 0168:fixme:file:FILE_GetSymlink MAGIC /home/ehoover/staging-50586/pfx/dosdevices/c:/Program Files/Black Tree Gaming Ltd/Vortex /home/ehoover/staging-50586/Vortex Mod Manager 0168:fixme:file:unix_to_nt_file_name /home/ehoover/staging-50586/Vortex Mod Manager: /Vortex Mod Manager (X:) === So, given that I can: 1) Fix this so that it "works" (reports the target properly) but gives you problems in terms of not properly finding everything. This is likely closest to what would happen on Windows if you set up the same situation with mklink. Is there any chance that you have a Windows box where you can verify that? 2) Report an unrecognized tag for unix symlinks (see hacky attachment, if you want to give that a try) 3) Hide the symlinks completely Zeb, do you have thoughts? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #35 from Arnaud Dovi <mr.dovi(a)gmail.com> --- (In reply to Erich E. Hoover from comment #34)
So, given that I can: 1) Fix this so that it "works" (reports the target properly) but gives you problems in terms of not properly finding everything. This is likely closest to what would happen on Windows if you set up the same situation with mklink. Is there any chance that you have a Windows box where you can verify that?
Nope unfortunately I have no Windows here -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #36 from Arnaud Dovi <mr.dovi(a)gmail.com> --- (In reply to Erich E. Hoover from comment #34)
That reproduces the error on startup really effectively, thanks. However, I don't know how to tell if I've fixed the (In reply to Arnaud Dovi from comment #33)
FYI, You can see the NtQueryInformationFile issue on Startup only if you replace the Vortex link to a common unix directory symlink, in the steps to reproduce I wrote it as a "MKLINK" so it does not crash. But for the second issue with a MKLINK link, you will notice in the loaddll entries that wine-staging 6.0 starts to use X:\ upon loading Electron modules, while on wine-6.0 it never uses X:\ but always C:\Program Files\...\Vortex -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #37 from Zebediah Figura <z.figura12(a)gmail.com> --- (In reply to Erich E. Hoover from comment #34)
Yes, thank you. I removed the internal uses where we get the link target, so I can confirm that this application is aware of reparse tags and that it is effectively responsible for the "problem": === 0168:fixme:file:FILE_GetSymlink MAGIC /home/ehoover/staging-50586/pfx/dosdevices/c:/Program Files/Black Tree Gaming Ltd/Vortex /home/ehoover/staging-50586/Vortex Mod Manager 0168:fixme:file:unix_to_nt_file_name /home/ehoover/staging-50586/Vortex Mod Manager: /Vortex Mod Manager (X:) ===
So, given that I can: 1) Fix this so that it "works" (reports the target properly) but gives you problems in terms of not properly finding everything. This is likely closest to what would happen on Windows if you set up the same situation with mklink. Is there any chance that you have a Windows box where you can verify that? 2) Report an unrecognized tag for unix symlinks (see hacky attachment, if you want to give that a try) 3) Hide the symlinks completely
Zeb, do you have thoughts?
Sorry, I'm having a bit of trouble understanding what's going wrong here. Can you please clarify the situation (is this a file or directory symlink, does it cross volumes, and how is the application misbehaving?) -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #38 from Arnaud Dovi <mr.dovi(a)gmail.com> --- If you prefer guy I will do another ticket to better track the other issue -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #39 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Zebediah Figura from comment #37)
... Sorry, I'm having a bit of trouble understanding what's going wrong here. Can you please clarify the situation (is this a file or directory symlink, does it cross volumes, and how is the application misbehaving?)
Sorry I didn't respond to this earlier, I thought I had and dropped it off my list. In this situation the user has replaced one of the application's directories with a directory symlink that points to another volume. It appears that the application sees the symlink and follows it, but that this breaks its ability to find some files that it treats relative to the launch directory. To use Arnaud's example: 1) application is at c:\symlink\app.exe 2) c:\symlink links to x:\appdir\ 3) application reads symlink and converts its path to x:\appdir\app.exe 4) application attempts to read files "up" a directory (..\myfolder\stuff) 5) application expects to find c:\myfolder\stuff 6) application fails to find it, because it's now trying to find x:\myfolder\stuff I'll see if I can find some time this weekend to dig out my old windows box, as I can probably "put this to bed" by recreating this behavior on windows with the same symlink situation. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #40 from Zebediah Figura <z.figura12(a)gmail.com> --- (In reply to Erich E. Hoover from comment #39)
(In reply to Zebediah Figura from comment #37)
... Sorry, I'm having a bit of trouble understanding what's going wrong here. Can you please clarify the situation (is this a file or directory symlink, does it cross volumes, and how is the application misbehaving?)
Sorry I didn't respond to this earlier, I thought I had and dropped it off my list. In this situation the user has replaced one of the application's directories with a directory symlink that points to another volume. It appears that the application sees the symlink and follows it, but that this breaks its ability to find some files that it treats relative to the launch directory.
To use Arnaud's example: 1) application is at c:\symlink\app.exe 2) c:\symlink links to x:\appdir\ 3) application reads symlink and converts its path to x:\appdir\app.exe 4) application attempts to read files "up" a directory (..\myfolder\stuff) 5) application expects to find c:\myfolder\stuff 6) application fails to find it, because it's now trying to find x:\myfolder\stuff
I'll see if I can find some time this weekend to dig out my old windows box, as I can probably "put this to bed" by recreating this behavior on windows with the same symlink situation.
Yeah, no problem, I've been a bit overwhelmed too... Is the application specifically reading the symlink, or is it just using GetFinalPathNameByHandle() and getting the realpath? (which IIRC that function does give the realpath, cf. a20d8bda3). Not that it matters; the application is broken either way. Unfortunately this seems difficult to work around without aggressively trying to hide symlinks. Problem is, we are going to run into more broken applications like this. Some symlink issues are easy enough to work around, but some like this are harder. But when I think about what would be necessary to really completely hide Unix directory symlinks, it just ends up feeling hacky one way or another; I can see it easily becoming an unmaintainable pile of workarounds. Maybe I'm being too paranoid, though... Sucks, but the best answer may be "you broke the application, you get to pick up the pieces" :-( -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #41 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Zebediah Figura from comment #40)
(In reply to Erich E. Hoover from comment #39)
... I'll see if I can find some time this weekend to dig out my old windows box, as I can probably "put this to bed" by recreating this behavior on windows with the same symlink situation.
Yeah, no problem, I've been a bit overwhelmed too...
Is the application specifically reading the symlink, or is it just using GetFinalPathNameByHandle() and getting the realpath? (which IIRC that function does give the realpath, cf. a20d8bda3). Not that it matters; the application is broken either way. Unfortunately this seems difficult to work around without aggressively trying to hide symlinks.
I didn't get around to my test this weekend, but when I get a chance I'll check both this and what happens in the same situation on Windows.
Problem is, we are going to run into more broken applications like this. Some symlink issues are easy enough to work around, but some like this are harder. But when I think about what would be necessary to really completely hide Unix directory symlinks, it just ends up feeling hacky one way or another; I can see it easily becoming an unmaintainable pile of workarounds. Maybe I'm being too paranoid, though...
Yeah, that's one of the reasons why I think it might be easier to hide unix symlinks rather than to treat them as Windows symlinks. Hiding them is _much_ easier than supporting them properly. As the patches stand, there's only "two" places that need to care: 1) server/fd.c: open_fd() would need to ignore FILE_OPEN_REPARSE_POINT on unix symlinks (any symlink with invalid reparse tag magic) 2) dlls/ntdll/unix/file.c get_file_info() and fd_get_file_info() would need to not report FILE_ATTRIBUTE_REPARSE_POINT for unix symlinks The only other thing that might be required, which is not difficult, would be to _not_ use the realpath() server/fd.c for regular unix symlinks and use a "normalized" path instead.
Sucks, but the best answer may be "you broke the application, you get to pick up the pieces" :-(
Yeah, if we try to report unix symlinks "properly" that is a very distinct possibility. (In reply to Arnaud Dovi from comment #38)
If you prefer guy I will do another ticket to better track the other issue
The original issue here should be fixed by staging commit 750044c08c49c7a117fcc911506a198969379144, so yeah - we should probably open a new issue. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #42 from Zebediah Figura <z.figura12(a)gmail.com> --- (In reply to Erich E. Hoover from comment #41)
Problem is, we are going to run into more broken applications like this. Some symlink issues are easy enough to work around, but some like this are harder. But when I think about what would be necessary to really completely hide Unix directory symlinks, it just ends up feeling hacky one way or another; I can see it easily becoming an unmaintainable pile of workarounds. Maybe I'm being too paranoid, though...
Yeah, that's one of the reasons why I think it might be easier to hide unix symlinks rather than to treat them as Windows symlinks. Hiding them is _much_ easier than supporting them properly. As the patches stand, there's only "two" places that need to care: 1) server/fd.c: open_fd() would need to ignore FILE_OPEN_REPARSE_POINT on unix symlinks (any symlink with invalid reparse tag magic) 2) dlls/ntdll/unix/file.c get_file_info() and fd_get_file_info() would need to not report FILE_ATTRIBUTE_REPARSE_POINT for unix symlinks
The only other thing that might be required, which is not difficult, would be to _not_ use the realpath() server/fd.c for regular unix symlinks and use a "normalized" path instead.
I think not using realpath() is absolutely required. We'd still have to manually resolve the real path for non-Unix symlinks, though. But here are the other problems: * What does GetVolumePathName() on a cross-device Unix symlink return? I'm not even necessarily thinking about file IDs here—we can fit st_dev + st_ino into a 16-byte file ID if we have to—but rather: * do we report that two files are on the same device when they aren't actually? Would it break an application that suddenly can't cope when renaming the file fails? * What if you then pass that name to other volume functions? If GetVolumePathName() returns the symlink, how does kernelbase determine the actual mount point? If it returns the C: drive for every file, then we lie to the application about aspects of the file system—say, the amount of free space left. * If we punt and pretend that there's only ever one file system—what about removable devices? Do we apply a heuristic and always expose cross-device host symlinks to removable devices, but not otherwise? Maybe there's an argument for hiding host symlinks only in the same-device case. On the other hand, I'm inclined to say that some applications are broken, and in that case the user will just have to work around that by not using symlinks. How many applications are actually affected by such symlink bugs is of course difficult or impossible to guess, but at least we haven't seen *that* many come up yet, not in wine-staging... -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #43 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Zebediah Figura from comment #42)
... I think not using realpath() is absolutely required. We'd still have to manually resolve the real path for non-Unix symlinks, though.
For NT symlinks I'm currently storing both the original/symlink path (non-normalized) and the realpath(), if you're saying that the original/symlink path needs to be normalized then I should probably add that to the current patches.
But here are the other problems:
* What does GetVolumePathName() on a cross-device Unix symlink return? I'm not even necessarily thinking about file IDs here—we can fit st_dev + st_ino into a 16-byte file ID if we have to—but rather:
We can easily do the same fake-out that we're doing right now with FileAttributeTagInformation returning IO_REPARSE_TAG_MOUNT_POINT. It's possible here that there are applications that would be broken by this (but they would be just as broken now), but worst case this could be replaced with a wine-specific option or FILE_INFORMATION_CLASS.
* do we report that two files are on the same device when they aren't actually? Would it break an application that suddenly can't cope when renaming the file fails?
Not unless we have to, since right now Wine doesn't do this with unix symlinks and nobody seems to notice.
* What if you then pass that name to other volume functions? If GetVolumePathName() returns the symlink, how does kernelbase determine the actual mount point? If it returns the C: drive for every file, then we lie to the application about aspects of the file system—say, the amount of free space left.
Since Wine doesn't do this right now, I would not suggest that an initial implementation do it either.
* If we punt and pretend that there's only ever one file system—what about removable devices? Do we apply a heuristic and always expose cross-device host symlinks to removable devices, but not otherwise?
I really don't think we should pretend there's only one file system.
Maybe there's an argument for hiding host symlinks only in the same-device case.
I think there's a decent argument that we could start from the current Wine position (hiding) and add complexity if it proves to be a problem.
On the other hand, I'm inclined to say that some applications are broken, and in that case the user will just have to work around that by not using symlinks. How many applications are actually affected by such symlink bugs is of course difficult or impossible to guess, but at least we haven't seen *that* many come up yet, not in wine-staging...
Yeah, I think we are likely to find more issues now that we've started reporting unix symlinks as NT symlinks (especially since Wine creates symlinks for My Documents, Desktop, and Downloads). Before we only had the possibility of applications seeing NT symlinks from Windows applications that actually created them in the first place. I'd be happy to be wrong though, code-wise reporting unix symlinks as NT symlinks is easier than hiding them ;) -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #44 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Arnaud Dovi from comment #38)
If you prefer guy I will do another ticket to better track the other issue
For when you get a chance to verify that this issue is fixed and open another ticket, I have an idea of something to try. It might be a bad idea, but it looks like these days there's now a special reparse tag in WSL for linux/unix symlinks (IO_REPARSE_TAG_LX_SYMLINK). This might be a way for us to "properly" report our unix symlinks without breaking this program (running your test case, I _think_ it works). -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 Erich E. Hoover <erich.e.hoover(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|UNCONFIRMED |RESOLVED Resolution|--- |FIXED --- Comment #45 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- Fixed in wine-staging 6.4. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #46 from Zebediah Figura <z.figura12(a)gmail.com> --- I think there may be some confusion. Currently the situation in upstream wine is: * NtQueryAttributesFile() reports FILE_ATTRIBUTE_REPARSE_POINT for directory symlinks, regardless of whether FILE_OPEN_REPARSE_POINT is specified. * FILE_OPEN_REPARSE_POINT never opens the symlink for NtCreateFile/NtOpenFile. * FSCTL_*_REPARSE_POINT is not supported on symlinks. * ObjectNameInformation and similar APIs report the resolved path for both file and directory symlinks. I don't know of any application offhand that is actually broken by these. (I do know applications that are broken by GetVolumePathName() not returning a drive root, but that's mostly orthogonal). The question is, does "hiding all Unix symlinks" mean roughly this (probably while also respecting FILE_OPEN_REPARSE_POINT in NtQueryAttributesFile)? Respecting FILE_OPEN_REPARSE_POINT in NtQueryAttributesFile but not NtCreateFile? Returning the target path from ObjectNameInformation? If an application passes FILE_OPEN_REPARSE_POINT or uses FSCTL codes, shouldn't it already know how to deal with the results? Based on comment 39, I'd think the part that's actually breaking *this* application is the fourth bullet point, viz. that ObjectNameInformation (=GetFinalPathNameByHandle) returns the resolved symlink, but I'm probably missing some context (or I've already forgotten it), because I don't know why that'd only break with recent symlink changes (I'd think it'd happen with upstream wine too?) But yeah, we can't easily lie about that, because GetVolumePathName() needs it. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #47 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Zebediah Figura from comment #46)
I think there may be some confusion.
Sorry, that's my bad. There's a lot of different pieces to this, so I apologize for making this difficult.
Currently the situation in upstream wine is: ... I don't know of any application offhand that is actually broken by these. (I do know applications that are broken by GetVolumePathName() not returning a drive root, but that's mostly orthogonal).
Agreed.
The question is, does "hiding all Unix symlinks" mean roughly this (probably while also respecting FILE_OPEN_REPARSE_POINT in NtQueryAttributesFile)? Respecting FILE_OPEN_REPARSE_POINT in NtQueryAttributesFile but not NtCreateFile? Returning the target path from ObjectNameInformation?
Roughly yes, when I say "hiding unix symlinks" I mean treating them as similar to upstream Wine as possible. In this case I would say that NtCreateFile with FILE_OPEN_REPARSE_POINT would check the symlink type and, if it's not a recognized reparse tag, ignore the FILE_OPEN_REPARSE_POINT flag. ObjectNameInformation would, similarly, need to return the target path.
Based on comment 39, I'd think the part that's actually breaking *this* application is the fourth bullet point, viz. that ObjectNameInformation (=GetFinalPathNameByHandle) returns the resolved symlink, but I'm probably missing some context (or I've already forgotten it), because I don't know why that'd only break with recent symlink changes (I'd think it'd happen with upstream wine too?) ...
This application actually uses FSCTL_GET_REPARSE_POINT to read the reparse point and (sorry, I thought I reported back on this). If the tag type is one of the normal ones then it interprets the path and uses that to do some voodoo that I have no visibility into. If you use an unrecognized tag type like IO_REPARSE_TAG_LX_SYMLINK then it appears to behave normally (though I would appreciate if Arnaud could double check that). -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #48 from Zebediah Figura <z.figura12(a)gmail.com> --- (In reply to Erich E. Hoover from comment #47)
Based on comment 39, I'd think the part that's actually breaking *this* application is the fourth bullet point, viz. that ObjectNameInformation (=GetFinalPathNameByHandle) returns the resolved symlink, but I'm probably missing some context (or I've already forgotten it), because I don't know why that'd only break with recent symlink changes (I'd think it'd happen with upstream wine too?) ...
This application actually uses FSCTL_GET_REPARSE_POINT to read the reparse point and (sorry, I thought I reported back on this). If the tag type is one of the normal ones then it interprets the path and uses that to do some voodoo that I have no visibility into. If you use an unrecognized tag type like IO_REPARSE_TAG_LX_SYMLINK then it appears to behave normally (though I would appreciate if Arnaud could double check that).
Wait, so it *specifically* requests the resolved target path and *still* treats it like it's not resolved? Are we sure we can't just resolve this as "terminally broken application"? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #49 from Erich E. Hoover <erich.e.hoover(a)gmail.com> --- (In reply to Zebediah Figura from comment #48)
... Wait, so it *specifically* requests the resolved target path and *still* treats it like it's not resolved?
This is an intermediate directory, so it _looks_ like what happens is that it's resolving the target of the directory and then using a relative path that's actually in the original location. Something like this: 1) application is at c:\symlink\app.exe 2) c:\symlink links to x:\appdir\ 3) application reads symlink and converts its path to x:\appdir\app.exe 4) application attempts to read files "up" a directory (..\myfolder\stuff) 5) application expects to find c:\myfolder\stuff 6) application fails to find it, because it's now trying to find x:\myfolder\stuff
Are we sure we can't just resolve this as "terminally broken application"?
Possibly, that's why I said that I'm pretty sure that this would also happen if you made a symlink out of this particular directory on Windows. I haven't had a chance to try and break out my old box, but I'll try to get to that again this weekend. My concern is that Wine users have become used to the unix symlinks being "hidden" and not having to worry about applications doing strange things like this. If we instead report the unix symlinks as being "hidden" then this wouldn't be a problem, since that's effectively what Wine currently does. Alternatively, giving these symlinks a custom reparse tag (or IO_REPARSE_TAG_LX_SYMLINK) is likely to work for most (all?) applications since the path that's returned is a native unix path that the application cannot possibly use. (In any case this particular bug is fixed, NtQueryInformationFile -> c0000003) -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #50 from Arnaud Dovi <mr.dovi(a)gmail.com> --- Thank you guys I confirm initial reported issue is fixed :) Sorry for long time reply was busy finishing my Steam guide, will link back it here since you are maybe gamers and steam lovers, I use simple instruction on how to install a famous app for use as non-Steam game with Proton GE (wine-staging 6.4) with specific instructions on how to bypass the use of non necessary third parties such as lutris, winetricks https://steamcommunity.com/sharedfiles/filedetails/?id=2386287653 But about the secondary issue, if you think that is not critical to correct, do not bother to correct it, I really do not know the impact of this problem, maybe it affects a lot of my in this case yes it would be smart to do that, but if I'm the only one who reported it, I found a cleaner way to do it which is to create a new dosdevice under X: -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 --- Comment #51 from Zebediah Figura <z.figura12(a)gmail.com> --- (In reply to Erich E. Hoover from comment #49)
(In reply to Zebediah Figura from comment #48)
... Wait, so it *specifically* requests the resolved target path and *still* treats it like it's not resolved?
This is an intermediate directory, so it _looks_ like what happens is that it's resolving the target of the directory and then using a relative path that's actually in the original location. Something like this: 1) application is at c:\symlink\app.exe 2) c:\symlink links to x:\appdir\ 3) application reads symlink and converts its path to x:\appdir\app.exe 4) application attempts to read files "up" a directory (..\myfolder\stuff) 5) application expects to find c:\myfolder\stuff 6) application fails to find it, because it's now trying to find x:\myfolder\stuff
Yeah, okay, that seems to be exactly what you said in comment 39, so I must be asking you to repeat yourself here. I just want to confirm that (a) the application is specifically aware of symlinks, (b) it's trying to read the symlink target specifically via FSCTL_GET_REPARSE_POINT, rather than querying ObjectNameInformation or something, (c) its symlink handling code is broken and uses the target path when it should use the source path. That is, I want to confirm that this is a bug in an application that tries to handle symlinks but does it incorrectly [and that it's not a bug with how we handle symlinks in wine-staging anywhere...] -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=50586 Zebediah Figura <z.figura12(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Fixed by SHA1| |750044c08c49c7a117fcc911506 | |a198969379144 Status|RESOLVED |CLOSED --- Comment #52 from Zebediah Figura <z.figura12(a)gmail.com> --- Closing bugs fixed in wine-staging 6.3. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
participants (1)
-
WineHQ Bugzilla