in the process of separating ntdll from kernel32, one of the next hurdle if the implementation of NtCreateFile. Since it's rather a core API, some thoughts are needed. Moreover, there are a few shortcomings in current file handling we might want to address.
The goal of this small RFC is to: 1/ gather all known shortcomings of current file implementation 2/ propose some solutions and roadmaps for the implementation
feel free to add your own remarks to these lists.
1/ shortcomings --------------- 1.1) no dynamicity in DOS drive management: user should be able, in his/her session(1), to either change the mapping of a DOS drive letter, or mount a SMB share somewhere...) 1.2) mounting should be done session wide and not process wide, and of course shared across processes 1.3) ntdll and kernel32 are not separated regarding file management 1.4) devices (vxd and DOS) management is still a hack 1.5) it may be necessary to introduce a notion of filesystem for some operations (open, read, write, seek, close, dir_enum, ioctl) in order to support several FS (unix, smb...) 1.6) poor management of ejectable devices 1.7) short/long file name conversion should be consistent (across several processes in the same session, across several sessions)
Note: (1) a session is linked to a running instance of the wineserver
2/ proposal -----------
2.1 "path gates" (target: 1.1, 1.2, 1.3/partly, 1.4/partly, 1.6/partly)
all file names will be handled using the NT-style: + DOS paths (C:\foo\bar) into ??\C:\foo\bar + network paths (\station\foo\bar) into ??\UNC\station\foo\bar + DOS device names (CON) into ??\CON + NT device names (\.\PhysicalDrive0) into ??\PhysicalDrive0 + a VXD ?? (normally \.\VFD) into ??\VFD
the device concept will disappear and be replaced by a "path gate": in NT-Win32 name space, point where to convert from a NT-path to a Unix-path
for example, assuming we have created a path-gate from \??\c: to /mnt/windows, the NT-path \??\c:\foo\bar will be transformed into the unix-path /mnt/windows/foo/bar
the path-gate will hold the current options that the device currently does (case sensivity, fixed/removable...)
path-gates will be stored in the server (they are session wide)
management of VxD and devices could benefit from it: they could be path gates pointing to them selves (or nothing ?)
2.2 FS (target: 1.4/partly, 1.5/partly, 1.6/partly)
I'm still wondering if we really need a FS (even a very simplistic one) in Wine
2.3 short/long file names (target 1.7)
since current short file name management seems satisfactory, there's no need IMO to move short/long names conversion in the server
Hello Eric,
the device concept will disappear and be replaced by a "path gate": in NT-Win32 name space, point where to convert from a NT-path to a Unix-path
for example, assuming we have created a path-gate from \??\c: to /mnt/windows, the NT-path \??\c:\foo\bar will be transformed into the unix-path /mnt/windows/foo/bar
It would be good to implement this "path gates" as near as possible like the way it is implemented in Windows NT/XP. This mapping is done using the NT object namespace. Object namespace is another virtual tree structure similar to a file system or the registry. It is accessed by functions like NtOpenDirectoryObject(), NtOpenSymbolicLinkObject(), NtQueryDirectoryObject(), NtQuerySymbolicLinkObject(), ... There has been a tool in the internet called "winobj.exe", with which you can examine this internal NT object namespace. However it is not compatible to XP, I don't know, if someone has yet updated it. But I do have a own program which can display it also for XP. Drive mapping is stored as symbolic links for example at "\GLOBAL??\D:". This links point to the partitions, like e.g. "\Device\HarddiskVolume3". There are also the ArcNames, which you can find in the NT bootloader config file "boot.ini". At "\ArcName\multi(0)disk(0)rdisk(0)" you can find another symbolic link to "\Device\Harddisk0\Partition0".
This examples are from my XP system. System versions before XP used a bit of another mapping schema.
By the way: You can also find the whole registry as a subtree in the NT object namespace at paths like "\REGISTRY\HKEY_LOCAL_MACHINE.... Would not be bad, if we could implement this also.
-- Martin Fuchs martin-fuchs@gmx.net
Martin Fuchs wrote:
Hello Eric,
Hi Martin thanks for the comments
the device concept will disappear and be replaced by a "path gate": in NT-Win32 name space, point where to convert from a NT-path to a Unix-path
for example, assuming we have created a path-gate from \??\c: to /mnt/windows, the NT-path \??\c:\foo\bar will be transformed into the unix-path /mnt/windows/foo/bar
It would be good to implement this "path gates" as near as possible like the way it is implemented in Windows NT/XP. This mapping is done using the NT object namespace. Object namespace is another virtual tree structure similar to a file system or the registry. It is accessed by functions like NtOpenDirectoryObject(), NtOpenSymbolicLinkObject(), NtQueryDirectoryObject(), NtQuerySymbolicLinkObject(), ... There has been a tool in the internet called "winobj.exe", with which you can examine this internal NT object namespace. However it is not compatible to XP, I don't know, if someone has yet updated it. But I do have a own program which can display it also for XP. Drive mapping is stored as symbolic links for example at "\GLOBAL??\D:". This links point to the partitions, like e.g. "\Device\HarddiskVolume3". There are also the ArcNames, which you can find in the NT bootloader config file "boot.ini". At "\ArcName\multi(0)disk(0)rdisk(0)" you can find another symbolic link to "\Device\Harddisk0\Partition0".
I do know how it's implemented in NT. anyway, there are a few differences between ROS and Wine: - in common, we do need to have the same interface as Windows do. This only applies to interface living in user land. - kernel land is another matter. ROS needs to stick to NT's behavior, Wine needs to convert calls to what Unix does provide. - ROS needs to mount physical drives, Wine makes (it's the current implementation, this is open to discussion) part of the Unix file hierarchy visible from the windows part
Regarding file support, the minimal things Wine need to do is: - support the same file naming as NT does (mainly the ??\ part of the NT name space), the rest is not absolutely needed. Currently, Wine does not implement a unique name space for all the objects (as NT does, using sub dir for some differenciation) but rather different name spaces - to allow the mapping from a NT path to a unix path, there are two simple ways to do it: => use all NT name space and somehow inject Unix file name space (/ and all the subdirs) in it => store the unix information (the path in the unix / hierarchy) as a specific attribute of the NT name space (at least the ??\ subtree)
I chose the second approach (you're more proposing the first one) because I felt it was more suited to wine behavior.
Concerning NTDLL's directory objects, I'm still wondering if we need (in Wine) to implement them. It's indeed needed in NTDLL's interface (object attribute for instance) and implements part of the "path gate" (especially some attributes like case sensivity in further lookups...). So, this is still part of the open questions I have (I may start without directory objects, and add them later on).
One of the strength of current NT name space is the orthogonality which is used across all objects in NTDLL (for example, in handle management, or sending requests - thru IRPs - at all the objects). The good side is to be able to convert an operation on a file into an IRP which is send to the FS which handles the file, which can in turn send it or transform it to another part... As wine implements a more monolithic approach, and since we don't have this orthoganility available today, I'm not yet convinced we need to stick more to the NT behavior than what I described above as the minimal requirements.
A+
Do we need attributes like hidden, system, etc? Feri.
On Tue, 19 Aug 2003 23:48:53 +0200 Ferenc Wagner wferi@afavant.elte.hu wrote:
Do we need attributes like hidden, system, etc? Feri.
More to the point we need attributes to allow support of NT ACLs. I've been looking at the problem and progressing only very slowly, some of the problems are technical, others are nothing to do with Wine at all.
At the level we are dealing with here the essential problems (for those who have not examined the issues) are :-
Not all supported O/S s offer ACLs.
Those that do support POSIX ACLs which do not map exactly to NT ACLs
Even in Systems which do support POSIX ACLs there is no guarantee that all disc filesystems at any one host will offer it, Of the Linux filesystem types only XFS supports ACLs out of the box. Work is in progress for ReiserFS and due to start for JFS. There is a patch for ext2/3. One map currently favoured is to use ext2/3/Reiser for the root filesystem and XFS for (some of) the rest).
All of the systems that do support ACLs use Extended Attributes to store them. I am still looking at limitations on number of EAs, but there is a possibility that an extreme case on NT may not be supported on Linux.
Since I've not yet got round to looking at TrustedBSD there's potentially another can of worms there.
My conclusion is that we do (regrettably) need a VFS layer to harmonise handling of all this lot and respond correctly when wine-ver is 9X.
Keith Matthews wrote:
On Tue, 19 Aug 2003 23:48:53 +0200 Ferenc Wagner wferi@afavant.elte.hu wrote:
Do we need attributes like hidden, system, etc? Feri.
More to the point we need attributes to allow support of NT ACLs. I've been looking at the problem and progressing only very slowly, some of the problems are technical, others are nothing to do with Wine at all.
At the level we are dealing with here the essential problems (for those who have not examined the issues) are :-
Not all supported O/S s offer ACLs.
Those that do support POSIX ACLs which do not map exactly to NT ACLs
Even in Systems which do support POSIX ACLs there is no guarantee that all disc filesystems at any one host will offer it, Of the Linux filesystem types only XFS supports ACLs out of the box. Work is in progress for ReiserFS and due to start for JFS. There is a patch for ext2/3. One map currently favoured is to use ext2/3/Reiser for the root filesystem and XFS for (some of) the rest).
All of the systems that do support ACLs use Extended Attributes to store them. I am still looking at limitations on number of EAs, but there is a possibility that an extreme case on NT may not be supported on Linux.
Since I've not yet got round to looking at TrustedBSD there's potentially another can of worms there.
My conclusion is that we do (regrettably) need a VFS layer to harmonise handling of all this lot and respond correctly when wine-ver is 9X.
I think Wine's goal is to provide this kind of feature of top of what the underlying OS provides (this is also the case for DOS HIDDEN and SYSTEM attributes => there are not available in standard Posix FS, so are managed by wine(*)). So IMO, file (protected by ACL) access should be provided by the underlying OS. ACL manipulation (from windows to posix - even if 1003.1e hasn't been voted by POSIX) should be added (I'm not sure we have a 1:1 mapping anyway) however, I don't understand your remark about the winver 9x... IMO, if the Linux user mounts a NT partition with ACL (and has the proper privileges) it should run Wine on this partition rather transparently (except for ACL manipulation function) it's not my goal to support: - real FS mounting in wine => this is the job of the OS, not wine's - a real VFS (as Linux does) - the (V)FS I'm talking about is more related to adaptation (as you mention), but I'm not sure I got you right
A+
(*) except files starting with a . which get the HIDDEN attribute
Ferenc Wagner wferi-at-afavant.elte.hu |Wine Mailing Lists| wrote:
Do we need attributes like hidden, system, etc? Feri.
Can't really answer your question, but on the kernel mailing list, There has been some discussion of whether these attributes should be exposed from mounted fat partitions: The fat fs maintainer has a patch, and is wondering what to do with it. (The patch exposes the fat attributes as extended attributes)
See the thread titled "[RFC] ioctl vs xattr for the filesystem specific attributes".
Hope this helps -Rob.
This is a re-post of a message I managed to get to Ferenc Wagner, but due to my stupidity, didn't get to the mailing list:
Ferenc Wagner wferi-at-afavant.elte.hu |Wine Mailing Lists| wrote:
Do we need attributes like hidden, system, etc? Feri.
Can't really answer your question, but on the kernel mailing list, There has been some discussion of whether these attributes should be exposed from mounted fat partitions: The fat fs maintainer has a patch, and is wondering what to do with it. (The patch exposes the fat attributes as extended attributes)
See the thread titled "[RFC] ioctl vs xattr for the filesystem specific attributes".
Hope this helps -Rob.
Can't really answer your question, but on the kernel mailing list, There has been some discussion of whether these attributes should be exposed from mounted fat partitions: The fat fs maintainer has a patch, and is wondering what to do with it. (The patch exposes the fat attributes as extended attributes)
See the thread titled "[RFC] ioctl vs xattr for the filesystem specific attributes".
yup that would help (to expose FAT attributes) my only remark is that we need to open the file in order to get its attributes I wonder if there would be an impact performance wise
-
Eric Pouech -
Ferenc Wagner -
Keith Matthews -
Martin Fuchs -
Robert North