[Bug 51059] New: Incorrect semantics of FILE_OPEN_REPARSE_POINT on Linux

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=51059

Bug ID: 51059
           Summary: Incorrect semantics of FILE_OPEN_REPARSE_POINT on
                    Linux
           Product: Wine-staging
           Version: 6.7
          Hardware: x86-64
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: -unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
                CC: [e-mail], [e-mail],
                    [e-mail]
      Distribution: ---

This bug affects the Cygwin installer (https://cygwin.com/setup-x86_64.exe);
specifically, it fails to verify SHA512 checksums of downloaded packages.

(Source at
https://cygwin.com/git/?p=cygwin-apps/setup.git;a=blob;f=package_source.cc,
 in method packagesource::check_sha512)

Running the installer with WINEDEBUG=trace+file,trace+server reports:

CreateFileW ("XXXX.tar.xz") [1]
    NtCreateFile ("XXXX.tar.xz", w/o FILE_OPEN_REPARSE_POINT) [1]
    CreateFileW return [1]
    NtQueryInformationFile FileBasicInformation [1]
    (server)close_handle [1]

NtCreateFile ("XXXX.tar.xz", w/ FILE_OPEN_REPARSE_POINT) [2]
    NtReadFile [2] -> STATUS_INVALID_HANDLE

The square brackets denote file handles. Handle reuse bug has been
ruled out.

This bug can be traced to an incorrect FILE_OPEN_REPARSE_POINT implementation
in Wine-staging, in
"ntdll-Junction_Points/0016-server-Implement-FILE_OPEN_REPARSE_POINT-option.patch":
> diff --git a/server/fd.c b/server/fd.c
> index 4f43f41fb31..a01d4c9c0f7 100644
> --- a/server/fd.c
> +++ b/server/fd.c
> @@ -107,6 +107,10 @@
>  #include "winioctl.h"
>  #include "ddk/wdm.h"
>
> +#if !defined(O_SYMLINK) && defined(O_PATH)
> +# define O_SYMLINK (O_NOFOLLOW | O_PATH)
> +#endif

On Linux, read/write/ioctl/... operations on an O_PATH descriptor will
fail with EBADF, which is then translated as STATUS_INVALID_HANDLE to the
application.
> -
>
> #if defined(HAVE_SYS_EPOLL_H) && defined(HAVE_EPOLL_CREATE)
>  # include <sys/epoll.h>
>  # define USE_EPOLL
> @@ -1958,6 +1962,11 @@ struct fd *open_fd( struct fd *root, const char *name, struct unicode_str nt_nam
>      }
>      else rw_mode = O_RDONLY;
>
> +#if defined(O_SYMLINK)
> - if ((options & FILE_OPEN_REPARSE_POINT) && !(flags & O_CREAT))
> -
>    flags |= O_SYMLINK;

Another difference is that O_PATH semantics override
O_RDONLY/O_WRONLY/O_RDWR, even if the target file isn't _actually_ a
symbolic link.

To properly implement it on Linux we should:

1. open with O_PATH.
2. check if the file is really a symlink (via fstat?).
3. if not, reopen via /proc/self/fd/%u (now w/o O_NOFOLLOW|O_PATH).

Note that this isn't an issue on XNU (macOS), where O_SYMLINK has an
equivalent semantics to FILE_OPEN_REPARSE_POINT.