April 2012 - Wine-Devel - list.winehq.org

Re: [PATCH 1/6] xstat: Add a pair of system calls to make extended file stats available

by Andreas Dilger

On 2012-04-26, at 6:51 PM, Dave Chinner wrote: > On Thu, Apr 26, 2012 at 02:32:36PM +0100, David Howells wrote: >> I wonder if there's a way to make this explicit - or is it something that if the bit isn't set, you can't use the value in st_blksize. >> I wonder if this value always has to be non-zero to make sure existing >> stat() doesn't explode. > > More likely it probably needs to be non-zero to prevent applications > doing division by block size from exploding... ;) Right, and any application which knows it needs the blocksize should also be requesting it when using the statxat() (or whatever) syscall. Cheers, Andreas

April 27, 2012

Re: [PATCH 0/6] Extended file stat system call

by Dave Chinner

On Thu, Apr 19, 2012 at 03:05:58PM +0100, David Howells wrote: > > Implement a pair of new system calls to provide extended and further extensible > stat functions. > > The second of the associated patches is the main patch that provides these new > system calls: > > ssize_t ret = xstat(int dfd, > const char *filename, > unsigned atflag, > unsigned mask, > struct xstat *buffer); > > ssize_t ret = fxstat(int fd, > unsigned atflag, > unsigned mask, > struct xstat *buffer); > > which are more fully documented in the first patch's description. > > These new stat functions provide a number of useful features, in summary: > > (1) More information: creation time, inode generation number, data version > number, flags/attributes. A subset of these is available through a number > of filesystems (such as CIFS, NFS, AFS, Ext4 and BTRFS). If we are adding per-inode flags, then what do we do with filesystem specific flags? e.g. XFS has quite a number of per-inode flags that don't align with any other filesystem (e.g. filestream allocator, real time file, behaviour inheritence flags, etc), but may be useful to retrieve in such a call. We currently have an ioctl to get that information from each inode. Have you thought about how to handle such flags? Along the same lines, filesytsems can have different allocation constraints to IO the filesystem block size - ext4 with it's bigalloc hack, XFS with it's per-inode extent size hints and the realtime device, etc. Then there's optimal IO characteristics (e.g. geometery hints like stripe unit/stripe width for the allocation policy of that given file) that applications could use if they were present rather than having to expose them through ioctls that nobody even knows about... Perhaps also exposing the project ID for quota purposes, like we do UID and GID. That way we wouldn't need a filesystem specific ioctl to read it.... Cheers, Dave. -- Dave Chinner david(a)fromorbit.com

April 27, 2012

Re: [PATCH 1/6] xstat: Add a pair of system calls to make extended file stats available

by Dave Chinner

On Thu, Apr 26, 2012 at 02:32:36PM +0100, David Howells wrote: > Andreas Dilger <adilger(a)dilger.ca> wrote: > > st_blksize may be variable for a distributed filesystem, It can be variable for local filesystems, too. XFS will vary the block size based on the configuration of the inode. e.g. if there is an extent allocation size hint on the inode, or it's on the realtime device, and so on. There is no guarantee that from file to file that it is constant. > I wonder if there's a way to make this explicit - or is it something that if > the bit isn't set, you can't use the value in st_blksize. > I wonder if this > value always has to be non-zero to make sure existing stat() doesn't explode. More likely it probably needs to be non-zero to prevent applications doing division by block size from exploding... ;) Cheers, Dave. -- Dave Chinner david(a)fromorbit.com

April 27, 2012

Re: [PATCH 0/6] Extended file stat system call

by Myklebust, Trond

On Thu, 2012-04-26 at 15:05 -0700, Roland McGrath wrote: > > What if the xstat() and struct xstat eventually becomes what userspace > > uses as stat() (as a wrapper) and struct stat (if such a thing is > > possible with glibc versioning)? > > It's certainly possible with symbol versioning, though it seems much more > likely that we'd stick with the existing struct stat and stat* interfaces > and only have the implementation using statx underneath (e.g. for new > machines or kernel ABIs where the kernel stops providing any calls except > for statxat), at least for the foreseeable future. > > > Do older programs that think they're using stat() and don't know about > > the extra fields available expect to see a useful value in st_ino? > > POSIX requires that st_ino have a useful value for the standard *stat calls. Yes, but we're talking about non-POSIX filesystems here. If the filesystem doesn't have a useful value for st_ino, then the usual way of dealing with those POSIX requirements is to fake up values. The question then becomes whether or not we care if it is the kernel or userland that fakes up those values. -- Trond Myklebust Linux NFS client maintainer NetApp Trond.Myklebust(a)netapp.com www.netapp.com

April 27, 2012

Re: [PATCH 0/6] Extended file stat system call

by Myklebust, Trond

On Thu, 2012-04-26 at 22:57 +0100, David Howells wrote: > Myklebust, Trond <Trond.Myklebust(a)netapp.com> wrote: > > > You are still not explaining why they need to know the values at all? If > > the values are bogus, then don't return them, and don't set the flag > > that says they are being returned. > th > What if the xstat() and struct xstat eventually becomes what userspace uses as > stat() (as a wrapper) and struct stat (if such a thing is possible with glibc > versioning)? Do older programs that think they're using stat() and don't know > about the extra fields available expect to see a useful value in st_ino? Does it really matter whether it is the kernel or userland that is responsible for faking up inode numbers? If userland wants to use xstat() in order to fake up a stat() call, then it gets to take responsibility for the results. -- Trond Myklebust Linux NFS client maintainer NetApp Trond.Myklebust(a)netapp.com www.netapp.com

April 27, 2012

Re: [PATCH 0/6] Extended file stat system call

by Andreas Dilger

On 2012-04-26, at 10:52, David Howells <dhowells(a)redhat.com> wrote: > Steve French <smfrench(a)gmail.com> wrote: >> >> Both NFS and CIFS (and SMB2) can return inode numbers or equivalent unique >> identifier, but in the case of CIFS some old servers don't support the calls >> which return inode numbers (or don't return them for all file system types, >> Windows FAT?) so in these cases cifs has to create inode numbers on the fly >> on the client. inode numbers created on the client are not "stable" they can >> change on unmount/remount (which can cause problems for backup applications). > > In the volatile case you'd probably want to unset XSTAT_INO in st_mask as the > inode number is a local fabrication. I'd agree. Why fake up an inode number if the application doesn't care? Most apps don't actually use the inode. The only uses I know for the inode number in userspace are backup, CIFS/NFS servers, and "ls -li" . > However, since there is a remote file ID, > we could add an XSTAT_INFO_FILE_ID flag to indicate there's a standard xattr > holding this. It is a bit strange that the kernel would return a flag that was not requested, but not fatal. > On CIFS this could be the servername + pathname, on NFS this > could be the server address + FH on AFS the cell+volID+FID+uniquifier for > example. That's independent of xstat, however, and wouldn't be returned as > it's a blob that could be quite large. > > I presume in some cases, there is not a unique file ID that persists across > rename. > >> Similarly NFSv4 does not require that servers always return stable inode >> numbers (that will never change) and introduced a concept of "volatile file >> handle." > > Can I presume the inode number cannot be considered stable if the NFS4 FH is > non-volatile? Furthermore, can I presume NFS2/3 inode numbers are supposed to > be stable? > >> Basically the question is whether it is worth reporting a flag on the call >> which returns the inode number to indicate that the inode number is "stable" >> (would not change on reboot or reconnection) or "volatile." Since the >> majority of NFS and SMB2 servers can return stable inode numbers, I don't >> feel strongly about the need for an indicator of "stable" vs. "volatile" but >> I mention it because backup and migration applications mention this (if inode >> numbers are volatile, they may have to check for hardlinks differently for >> example) > > It may be that unsetting XSTAT_INO if you've fabricated the inode number > locally is sufficient. > >>>>> Handle remote filesystems being offline and indicate this with >>>>> XSTAT_INFO_OFFLINE. >>>> >>>> You already have support for an indicator for offline files (HSM), > > Which indicator is this? Or do you mean XSTAT_INFO_OFFLINE? > >>>> would XSTAT_INFO_OFFLINE be intended for the case >>>> where the network session to the server is disconnected >>>> (and in which you case the application does not want to reconnect)? >>> >>> Hmmm... Interesting question. Both NTFS and CIFS have an offline >>> attribute (which is where I originally got this from) - but should I have a >>> separate indicator to indicate the client can't access a server over a >>> network (ie. we've gone to disconnected operation on this file)? >>> E.g. should there be a XSTAT_INFO_DISCONNECTED too? >> >> my reaction is no, since it adds complexity. If you do a stat on a >> disconnected volume (where the network is temporarily down) reconnection will >> be attempted. If reconnection fails then the xstat will either fail or be >> retried forever depending on the value of "hard" vs. "soft" mount flag. > > I was thinking of how to handle disconnected operation, where you can't just > sit there and churn waiting for the server to come back or give an error. On > the other hand, as long as there's some spare space in the struct, we can deal > with that later when we actually start to implement D/O. > > David > -- > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > the body of a message to majordomo(a)vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html

April 27, 2012

Re: [PATCH 0/6] Extended file stat system call

by Nix

On 26 Apr 2012, Roland McGrath verbalised: >> fileinfoat() perhaps? I think stat*at() is better, though, as people are >> used to the stat function family. > > Names like that were all I had thought of when I said I hadn't thought of > any good ones. ;-) Quite. It's a garden-path function name. "What's a foat and why would I want to put a file in one?" -- NULL && (void)

April 26, 2012

Re: [PATCH 0/6] Extended file stat system call

by Roland McGrath

> What if the xstat() and struct xstat eventually becomes what userspace > uses as stat() (as a wrapper) and struct stat (if such a thing is > possible with glibc versioning)? It's certainly possible with symbol versioning, though it seems much more likely that we'd stick with the existing struct stat and stat* interfaces and only have the implementation using statx underneath (e.g. for new machines or kernel ABIs where the kernel stops providing any calls except for statxat), at least for the foreseeable future. > Do older programs that think they're using stat() and don't know about > the extra fields available expect to see a useful value in st_ino? POSIX requires that st_ino have a useful value for the standard *stat calls. Thanks, Roland

April 26, 2012

Re: [PATCH 0/6] Extended file stat system call

by Roland McGrath

> fileinfoat() perhaps? I think stat*at() is better, though, as people are > used to the stat function family. Names like that were all I had thought of when I said I hadn't thought of any good ones. ;-)

April 26, 2012

Re: [PATCH 0/6] Extended file stat system call

by David Howells

Myklebust, Trond <Trond.Myklebust(a)netapp.com> wrote: > You are still not explaining why they need to know the values at all? If > the values are bogus, then don't return them, and don't set the flag > that says they are being returned. What if the xstat() and struct xstat eventually becomes what userspace uses as stat() (as a wrapper) and struct stat (if such a thing is possible with glibc versioning)? Do older programs that think they're using stat() and don't know about the extra fields available expect to see a useful value in st_ino? David

April 26, 2012