https://bugs.winehq.org/show_bug.cgi?id=51059
Bug ID: 51059 Summary: Incorrect semantics of FILE_OPEN_REPARSE_POINT on Linux Product: Wine-staging Version: 6.7 Hardware: x86-64 OS: Linux Status: UNCONFIRMED Severity: normal Priority: P2 Component: -unknown Assignee: wine-bugs@winehq.org Reporter: jinoh.kang.kr@gmail.com CC: erich.e.hoover@gmail.com, leslie_alistair@hotmail.com, z.figura12@gmail.com Distribution: ---
This bug affects the Cygwin installer (https://cygwin.com/setup-x86_64.exe); specifically, it fails to verify SHA512 checksums of downloaded packages.
(Source at https://cygwin.com/git/?p=cygwin-apps/setup.git;a=blob;f=package_source.cc, in method packagesource::check_sha512)
Running the installer with WINEDEBUG=trace+file,trace+server reports:
CreateFileW ("XXXX.tar.xz") [1] NtCreateFile ("XXXX.tar.xz", w/o FILE_OPEN_REPARSE_POINT) [1] CreateFileW return [1] NtQueryInformationFile FileBasicInformation [1] (server)close_handle [1]
NtCreateFile ("XXXX.tar.xz", w/ FILE_OPEN_REPARSE_POINT) [2] NtReadFile [2] -> STATUS_INVALID_HANDLE
The square brackets denote file handles. Handle reuse bug has been ruled out.
This bug can be traced to an incorrect FILE_OPEN_REPARSE_POINT implementation in Wine-staging, in "ntdll-Junction_Points/0016-server-Implement-FILE_OPEN_REPARSE_POINT-option.patch":
diff --git a/server/fd.c b/server/fd.c index 4f43f41fb31..a01d4c9c0f7 100644 --- a/server/fd.c +++ b/server/fd.c @@ -107,6 +107,10 @@ #include "winioctl.h" #include "ddk/wdm.h"
+#if !defined(O_SYMLINK) && defined(O_PATH) +# define O_SYMLINK (O_NOFOLLOW | O_PATH) +#endif
On Linux, read/write/ioctl/... operations on an O_PATH descriptor will fail with EBADF, which is then translated as STATUS_INVALID_HANDLE to the application.
#if defined(HAVE_SYS_EPOLL_H) && defined(HAVE_EPOLL_CREATE) # include <sys/epoll.h> # define USE_EPOLL @@ -1958,6 +1962,11 @@ struct fd *open_fd( struct fd *root, const char *name, struct unicode_str nt_nam } else rw_mode = O_RDONLY;
+#if defined(O_SYMLINK)
- if ((options & FILE_OPEN_REPARSE_POINT) && !(flags & O_CREAT))
flags |= O_SYMLINK;
Another difference is that O_PATH semantics override O_RDONLY/O_WRONLY/O_RDWR, even if the target file isn't _actually_ a symbolic link.
To properly implement it on Linux we should:
1. open with O_PATH. 2. check if the file is really a symlink (via fstat?). 3. if not, reopen via /proc/self/fd/%u (now w/o O_NOFOLLOW|O_PATH).
Note that this isn't an issue on XNU (macOS), where O_SYMLINK has an equivalent semantics to FILE_OPEN_REPARSE_POINT.
https://bugs.winehq.org/show_bug.cgi?id=51059
jinoh.kang.kr@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.winehq.org/sho | |w_bug.cgi?id=50036
https://bugs.winehq.org/show_bug.cgi?id=51059
--- Comment #1 from Erich E. Hoover erich.e.hoover@gmail.com --- You are correct that O_SYMLINK is not identical to (O_NOFOLLOW | O_PATH), but I did some testing of this some time ago and the differences don't really matter for implementing FILE_OPEN_REPARSE_POINT. What you get currently in wine-staging, as you have discovered, is an error on read/write/etc. where the real behavior is that these succeed but do nothing (return no bytes, size 0, etc.). I have not gone to the trouble of implementing this because Windows applications don't expect this to "work".
However, you have stumbled across something _very_ interesting here in that it appears that cygwin opens the file and then "changes" it when converting it to a unix handle: https://cygwin.com/git/?p=cygwin-apps/setup.git;a=blob;f=filemanip.cc;h=48f5...
I'll need to put together a test case, but I suspect that _open_osfhandle actually opens a new handle that doesn't have FILE_OPEN_REPARSE_POINT set. I can probably get to that next weekend.
https://bugs.winehq.org/show_bug.cgi?id=51059
--- Comment #2 from Jinoh Kang jinoh.kang.kr@gmail.com --- (In reply to Erich E. Hoover from comment #1)
You are correct that O_SYMLINK is not identical to (O_NOFOLLOW | O_PATH), but I did some testing of this some time ago and the differences don't really matter for implementing FILE_OPEN_REPARSE_POINT.
What you get currently in wine-staging, as you have discovered, is an error on read/write/etc. where the real behavior is that these succeed but do nothing (return no bytes, size 0, etc.).
Or Is it so? I actually based my claim of this behavior being a bug based on Microsoft Docs on CreateFileW:
*FILE_FLAG_OPEN_REPARSE_POINT* 0x00200000
Normal reparse point processing will not occur; CreateFile will attempt to open the reparse point. When a file is opened, a file handle is returned, whether or not the filter that controls the reparse point is operational.
This flag cannot be used with the CREATE_ALWAYS flag.
If the file is not a reparse point, then this flag is ignored.
I acknowledge the docs (or MSDN) hasn't exactly been known for its accuracy on "internal" matters, though. Also there's no guarantee that the Win32 API won't do some other magic for that flag before passing it as FILE_OPEN_REPARSE_POINT to NtCreateFile. Might test about this later.
I have not gone to the trouble of implementing this because Windows applications don't expect this to "work".
However, you have stumbled across something _very_ interesting here in that it appears that cygwin opens the file and then "changes" it when converting it to a unix handle: https://cygwin.com/git/?p=cygwin-apps/setup.git;a=blob;f=filemanip.cc; h=48f5117f6e4c8a2f3bd13bc4c3c22ac418634ede;hb=HEAD#l469
I see it was the setup part. I couldn't debug it myself ATM since the debugger for some reason tripped into assertion failure while parsing DWARF2 debugging symbols.
I noticed my transcription error on the trace (my apologies), so the first part of the trace actually goes:
CreateFileW ("XXXX.tar.xz") [1] NtCreateFile ("XXXX.tar.xz", w/o FILE_OPEN_REPARSE_POINT) [1] CreateFileW return [1] NtQueryInformationFile *FileStandardInformation* [1] (server)close_handle [1]
Now that it's known that the setup calls into NTDLL directly, I suppose this part was simply a pre-flight query to fetch the file existence or size.
NtCreateFile ("XXXX.tar.xz", w/ FILE_OPEN_REPARSE_POINT) [2] NtReadFile [2] -> STATUS_INVALID_HANDLE
So this one is from io_stream ctor -> nt_wfopen. I don't think _open_osfhandle should be doing anything on the handle, but this is a speculation.
I'll need to put together a test case, but I suspect that _open_osfhandle actually opens a new handle that doesn't have FILE_OPEN_REPARSE_POINT set. I can probably get to that next weekend.
https://bugs.winehq.org/show_bug.cgi?id=51059
--- Comment #3 from Erich E. Hoover erich.e.hoover@gmail.com --- (In reply to Jinoh Kang from comment #2)
(In reply to Erich E. Hoover from comment #1)
You are correct that O_SYMLINK is not identical to (O_NOFOLLOW | O_PATH), but I did some testing of this some time ago and the differences don't really matter for implementing FILE_OPEN_REPARSE_POINT.
What you get currently in wine-staging, as you have discovered, is an error on read/write/etc. where the real behavior is that these succeed but do nothing (return no bytes, size 0, etc.).
Or Is it so? I actually based my claim of this behavior being a bug based on Microsoft Docs on CreateFileW:
It's entirely possible my original test was flawed or I misinterpreted it, so I'll double check. I would have put the symlink test next to the real file test, so it's entirely possible that I misread the results. As you have pointed out, it is very easy to fix if that's the case.
... So this one is from io_stream ctor -> nt_wfopen. I don't think _open_osfhandle should be doing anything on the handle, but this is a speculation. ...
Since this routine can change the "mode" of the handle (append, read-only, etc.), I would not be surprised if it effectively "reopens" it with different flags. This would be harder to fix, so I'm actually hoping that I screwed up my original test on the behavior with real files.
https://bugs.winehq.org/show_bug.cgi?id=51059
--- Comment #4 from Erich E. Hoover erich.e.hoover@gmail.com --- (In reply to Erich E. Hoover from comment #3)
(In reply to Jinoh Kang from comment #2)
(In reply to Erich E. Hoover from comment #1)
You are correct that O_SYMLINK is not identical to (O_NOFOLLOW | O_PATH), but I did some testing of this some time ago and the differences don't really matter for implementing FILE_OPEN_REPARSE_POINT.
... Or Is it so? I actually based my claim of this behavior being a bug based on Microsoft Docs on CreateFileW:
It's entirely possible my original test was flawed or I misinterpreted it, so I'll double check. I would have put the symlink test next to the real file test, so it's entirely possible that I misread the results. As you have pointed out, it is very easy to fix if that's the case.
Looks like my original test was a mistake, I retested with FILE_OPEN_REPARSE_POINT on a regular file and the file behaved normally. I'll try to put together an update with this and some other fixes as soon as I can.
https://bugs.winehq.org/show_bug.cgi?id=51059
--- Comment #5 from Erich E. Hoover erich.e.hoover@gmail.com --- This should be fixed by staging commit 8e5c8cc63b57dbe9b2cf2ff075c12648845b22fa, would you mind retesting?
https://bugs.winehq.org/show_bug.cgi?id=51059
Jinoh Kang jinoh.kang.kr@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|UNCONFIRMED |RESOLVED
--- Comment #6 from Jinoh Kang jinoh.kang.kr@gmail.com --- Sorry for late update. Thanks a lot!
https://bugs.winehq.org/show_bug.cgi?id=51059
Erich E. Hoover erich.e.hoover@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |CLOSED
--- Comment #7 from Erich E. Hoover erich.e.hoover@gmail.com --- (In reply to Jinoh Kang from comment #6)
Sorry for late update. Thanks a lot!
Closing bug, fixed in wine-staging 6.10
-
WineHQ Bugzilla