Re: [2/3] ntdll: don't treat DOS paths starting with / as Unix paths
On Monday 06 April 2009 6:58:31 pm Vincent Povirk wrote:
On Mon, Apr 6, 2009 at 8:53 PM, Chris Robinson <chris.kcat(a)gmail.com> wrote:
Isn't that the python test's fault, then? Why can't c:\bin\sh exist?
On a real windows system, it won't.
It won't by default, but can't a user create it?
On Mon, Apr 6, 2009 at 9:15 PM, Chris Robinson <chris.kcat(a)gmail.com> wrote:
On Monday 06 April 2009 6:58:31 pm Vincent Povirk wrote:
On Mon, Apr 6, 2009 at 8:53 PM, Chris Robinson <chris.kcat(a)gmail.com> wrote:
Isn't that the python test's fault, then? Why can't c:\bin\sh exist?
On a real windows system, it won't.
It won't by default, but can't a user create it?
A real user who is trying to get real work done won't.
2009/4/7 Vincent Povirk <madewokherd+8cd9(a)gmail.com>:
On Mon, Apr 6, 2009 at 9:15 PM, Chris Robinson <chris.kcat(a)gmail.com> wrote:
On Monday 06 April 2009 6:58:31 pm Vincent Povirk wrote:
On Mon, Apr 6, 2009 at 8:53 PM, Chris Robinson <chris.kcat(a)gmail.com> wrote:
Isn't that the python test's fault, then? Why can't c:\bin\sh exist?
On a real windows system, it won't.
It won't by default, but can't a user create it?
A real user who is trying to get real work done won't.
Out of curiosity, why do the python tests even care if /bin/sh exists?
"Vincent Povirk" <madewokherd+8cd9(a)gmail.com> wrote:
Isn't that the python test's fault, then? Why can't c:\bin\sh exist?
On a real windows system, it won't.
It won't by default, but can't a user create it?
A real user who is trying to get real work done won't.
A real user who is trying to get real work done won't run the python test suite. -- Dmitry.
On Tue, Apr 7, 2009 at 5:43 AM, Dmitry Timoshkov <dmitry(a)codeweavers.com> wrote:
A real user who is trying to get real work done won't run the python test suite.
True. On Tue, Apr 7, 2009 at 4:45 AM, Henri Verbeet <hverbeet(a)gmail.com> wrote:
Out of curiosity, why do the python tests even care if /bin/sh exists?
The python tests run on multiple platforms, some of which have a working /bin/sh. If it exists, certain ones (there are only two) will start an sh process and read its output. Vincent Povirk
2009/4/8 Vincent Povirk <madewokherd+8cd9(a)gmail.com>:
On Tue, Apr 7, 2009 at 5:43 AM, Dmitry Timoshkov <dmitry(a)codeweavers.com> wrote:
A real user who is trying to get real work done won't run the python test suite.
True.
On Tue, Apr 7, 2009 at 4:45 AM, Henri Verbeet <hverbeet(a)gmail.com> wrote:
Out of curiosity, why do the python tests even care if /bin/sh exists?
The python tests run on multiple platforms, some of which have a working /bin/sh. If it exists, certain ones (there are only two) will start an sh process and read its output.
Surely this is still a bug in the Python test suite though. They should know that "/bin/sh" is not technically a valid path on Windows/DOS systems, and it only works due to system magic translating /'s to \'s.
On Tue, Apr 7, 2009 at 9:30 AM, Ben Klein <shacklein(a)gmail.com> wrote:
2009/4/8 Vincent Povirk <madewokherd+8cd9(a)gmail.com>:
On Tue, Apr 7, 2009 at 5:43 AM, Dmitry Timoshkov <dmitry(a)codeweavers.com> wrote:
A real user who is trying to get real work done won't run the python test suite.
True.
On Tue, Apr 7, 2009 at 4:45 AM, Henri Verbeet <hverbeet(a)gmail.com> wrote:
Out of curiosity, why do the python tests even care if /bin/sh exists?
The python tests run on multiple platforms, some of which have a working /bin/sh. If it exists, certain ones (there are only two) will start an sh process and read its output.
Surely this is still a bug in the Python test suite though. They should know that "/bin/sh" is not technically a valid path on Windows/DOS systems, and it only works due to system magic translating /'s to \'s.
"Working" in this case means that either: * os.exists('/bin/sh') returns False (the case on Windows) or * Python can create a /bin/sh process and open a pipe to it (the case on Linux) This doesn't depend on odd path translation in Windows/DOS. The tests break on Wine because neither of those are true. Vincent Povirk
On Tue, Apr 7, 2009 at 9:41 AM, Vincent Povirk <madewokherd+8cd9(a)gmail.com> wrote:
"Working" in this case means that either: * os.exists('/bin/sh') returns False (the case on Windows)
Do I understand you correctly to mean that if I create "C:\bin\sh" on windows and run the Python test suite, it will fail? -- -Austin
On Tue, Apr 7, 2009 at 10:35 AM, Austin English <austinenglish(a)gmail.com> wrote:
Do I understand you correctly to mean that if I create "C:\bin\sh" on windows and run the Python test suite, it will fail?
I haven't tried that, but I believe it will.
On Tue, Apr 7, 2009 at 10:39 AM, Vincent Povirk <madewokherd+8cd9(a)gmail.com> wrote:
On Tue, Apr 7, 2009 at 10:35 AM, Austin English <austinenglish(a)gmail.com> wrote:
Do I understand you correctly to mean that if I create "C:\bin\sh" on windows and run the Python test suite, it will fail?
I haven't tried that, but I believe it will.
Then as was already said, it's a Python bug, and should be fixed there. What if some other app actually depends on that behavior (mingw/cygwin perhaps)? -- -Austin
On Tuesday 07 April 2009 17:30:57 Ben Klein wrote:
2009/4/8 Vincent Povirk <madewokherd+8cd9(a)gmail.com>:
On Tue, Apr 7, 2009 at 5:43 AM, Dmitry Timoshkov <dmitry(a)codeweavers.com> wrote:
A real user who is trying to get real work done won't run the python test suite.
True.
On Tue, Apr 7, 2009 at 4:45 AM, Henri Verbeet <hverbeet(a)gmail.com> wrote:
Out of curiosity, why do the python tests even care if /bin/sh exists?
The python tests run on multiple platforms, some of which have a working /bin/sh. If it exists, certain ones (there are only two) will start an sh process and read its output.
Surely this is still a bug in the Python test suite though. They should know that "/bin/sh" is not technically a valid path on Windows/DOS systems, and it only works due to system magic translating /'s to \'s.
Ben, Paths with slashes ARE valid in DOS/Windows since the dawn of DOS 2.0 (first to include paths). While there are bugs in some versions at the API level forward or backward slashes (or mixtures) can be used for any operation. Only the command line parsers make a difference because forward slash was used as option separator since CP/M. So /bin/sh is just equivalent to \bin\sh on the default drive.
2009/4/8 Paul Chitescu <paulc(a)voip.null.ro>:
On Tuesday 07 April 2009 17:30:57 Ben Klein wrote:
2009/4/8 Vincent Povirk <madewokherd+8cd9(a)gmail.com>:
On Tue, Apr 7, 2009 at 5:43 AM, Dmitry Timoshkov <dmitry(a)codeweavers.com> wrote:
A real user who is trying to get real work done won't run the python test suite.
True.
On Tue, Apr 7, 2009 at 4:45 AM, Henri Verbeet <hverbeet(a)gmail.com> wrote:
Out of curiosity, why do the python tests even care if /bin/sh exists?
The python tests run on multiple platforms, some of which have a working /bin/sh. If it exists, certain ones (there are only two) will start an sh process and read its output.
Surely this is still a bug in the Python test suite though. They should know that "/bin/sh" is not technically a valid path on Windows/DOS systems, and it only works due to system magic translating /'s to \'s.
Ben,
Paths with slashes ARE valid in DOS/Windows since the dawn of DOS 2.0 (first to include paths). While there are bugs in some versions at the API level forward or backward slashes (or mixtures) can be used for any operation.
Only the command line parsers make a difference because forward slash was used as option separator since CP/M.
So /bin/sh is just equivalent to \bin\sh on the default drive.
This is what I meant about magic translation. It *shouldn't* work, but I'm aware that it does. DOS/Windows uses backslash as the delimiter when reporting and storing paths. Is the behaviour of magic translation from foreslash to backslash documented (by Microsoft) anywhere? I don't think there's any way for Wine to solve this ambiguity. Python's test suite would need to be fixed.
Hi! [I'm not subscribed, I hope I got the forged in-reply-to header right] Ben Klein wrote:
This is what I meant about magic translation. It *shouldn't* work, but I'm aware that it does. DOS/Windows uses backslash as the delimiter when reporting and storing paths. Is the behaviour of magic translation from foreslash to backslash documented (by Microsoft) anywhere?
From: http://msdn.microsoft.com/en-us/library/aa365247.aspx + MSDN Library + Win32 and COM Development + System Services + File Services + File Systems + File Management + About File Management + Creating, Deleting, and Maintaining Files * File Names, Paths, and Namespaces "Note File I/O functions in the Windows API convert "/" to "\" as part of converting the name to an NT-style name, except when using the "\\?\" prefix as detailed in the following sections." So, the Win32 API supports / BTW, people using the identity mount technique with MinGW (*) are not unlikely to have a /bin/sh on the current drive. Hmmm, but that's actually /bin/sh.exe so maybe they are safe? And I guess building mingw-gcc on Windows isn't "real work" (that's the main reason for the identiy mount, IIUC). Anyway, that's just another stab at the python test suite. Building stuff using MSYS/MinGW (or Cygwin) inside wine which requires an identity mount is surely just for fun. Anything MSYS or Cygwin inside Wine is for fun. The identity mount technique clearly does not work with Wine. Cheers, Peter (*) Google: mingw "identity mount"
2009/4/8 Ben Klein <shacklein(a)gmail.com>
This is what I meant about magic translation. It *shouldn't* work, but I'm aware that it does. DOS/Windows uses backslash as the delimiter when reporting and storing paths. Is the behaviour of magic translation from foreslash to backslash documented (by Microsoft) anywhere?
Microsoft's Larry Osterman blogs about it here http://blogs.msdn.com/larryosterman/archive/2005/06/24/432386.aspx "The DOS developers weren't particularly happy about [the choice of path character] so they coded the OS to accept either "/" or "\" character as the path character (this continues today, btw - try typing "notepad c:/boot.ini" on an XP machine (if you're an admin)). And they went one step further. They added an undocumented system call to change the switch character. And updated the utilities to respect this flag. And then they went and finished out the scenario: They added a config.sys option, SWITCHAR= that would let you set the switch character to "-". Which flipped MS-DOS into a *nix style system where command lines used "-switch", and paths were / delimited. I don't know the fate of the switchar API, it's been long gone for many years now." At what level this translation takes place - whether it's something in the command interpreter or further down into the API - I don't know. -- Chris
Am Mittwoch, den 08.04.2009, 09:11 +0100 schrieb Chris Howe:
I don't know the fate of the switchar API, it's been long gone for many years now."
At what level this translation takes place - whether it's something in the command interpreter or further down into the API - I don't know.
The DOS kernel accepts both '/' and '\' as path separator. The switchchar API was to be used by the application programs' internal command line parser, but near to nobody did it. The possibility to switch to unix semantics in this regard (introduced in DOS 2.0) failed as well as the option to make dos devices only available in the pseudo directory \DEV. IIRC some DOS 3.x made the kernel call that should enable the \DEV-enforcing to a NOP. There are a some prominent user space programs that used '/' in API calls. IIRC pkzip was one of them. Microsoft never broke this behaviour. And even if Wine would decide this behaviour to be a bug, we want it to be in Wine as Wine should be bug-to-bug compatible. Regards, Michael Karcher
Hi! [I just subscribed, I hope I got the forged in-reply-to header right] Ben Klein wrote:
This is what I meant about magic translation. It *shouldn't* work, but I'm aware that it does. DOS/Windows uses backslash as the delimiter when reporting and storing paths. Is the behaviour of magic translation from foreslash to backslash documented (by Microsoft) anywhere?
From: http://msdn.microsoft.com/en-us/library/aa365247.aspx + MSDN Library + Win32 and COM Development + System Services + File Services + File Systems + File Management + About File Management + Creating, Deleting, and Maintaining Files * File Names, Paths, and Namespaces "Note File I/O functions in the Windows API convert "/" to "\" as part of converting the name to an NT-style name, except when using the "\\?\" prefix as detailed in the following sections." So, the Win32 API supports / BTW, people using the identity mount technique with MinGW (*) are not unlikely to have a /bin/sh on the current drive. Hmmm, but that's actually /bin/sh.exe so maybe they are safe? And I guess building mingw-gcc on Windows isn't "real work" (that's the main reason for the identiy mount, IIUC). Anyway, that's just another stab at the python test suite. Building stuff using MSYS/MinGW (or Cygwin) inside wine which requires an identity mount is surely just for fun. Anything MSYS or Cygwin inside Wine is for fun. The identity mount technique clearly does not work with Wine. Cheers, Peter (*) Google: mingw "identity mount"
2009/4/8 Peter Rosin <peda(a)lysator.liu.se>:
Hi!
[I just subscribed, I hope I got the forged in-reply-to header right]
Ben Klein wrote:
This is what I meant about magic translation. It *shouldn't* work, but I'm aware that it does. DOS/Windows uses backslash as the delimiter when reporting and storing paths. Is the behaviour of magic translation from foreslash to backslash documented (by Microsoft) anywhere?
From: http://msdn.microsoft.com/en-us/library/aa365247.aspx
+ MSDN Library + Win32 and COM Development + System Services + File Services + File Systems + File Management + About File Management + Creating, Deleting, and Maintaining Files * File Names, Paths, and Namespaces
"Note File I/O functions in the Windows API convert "/" to "\" as part of converting the name to an NT-style name, except when using the "\\?\" prefix as detailed in the following sections."
So, the Win32 API supports /
BTW, people using the identity mount technique with MinGW (*) are not unlikely to have a /bin/sh on the current drive. Hmmm, but that's actually /bin/sh.exe so maybe they are safe? And I guess building mingw-gcc on Windows isn't "real work" (that's the main reason for the identiy mount, IIUC). Anyway, that's just another stab at the python test suite.
Building stuff using MSYS/MinGW (or Cygwin) inside wine which requires an identity mount is surely just for fun. Anything MSYS or Cygwin inside Wine is for fun. The identity mount technique clearly does not work with Wine.
Cheers, Peter
(*) Google: mingw "identity mount"
This is probably a really dumb question... but why does wine support UNIX paths? What is the circumstance where a Windows application will be trying to access a native file or directory? The only example I can think of is that an app has specifically been written to be used in Wine, in which case, shouldn't native UNIX paths be disabled by default, and perhaps turned on with an environment variable? Luke.
On Wed, Apr 8, 2009 at 10:07 AM, Luke Benstead <kazade(a)gmail.com> wrote:
This is probably a really dumb question... but why does wine support UNIX paths? What is the circumstance where a Windows application will be trying to access a native file or directory? The only example I can think of is that an app has specifically been written to be used in Wine, in which case, shouldn't native UNIX paths be disabled by default, and perhaps turned on with an environment variable?
If an app is written for use in Wine, there's an API it can use specifically for converting Unix paths to Windows paths. I have no idea why this is deemed useful. Then again, I'm also the guy who at the start of the thread said this is stupid and we really should stop. Vincent Povirk
On Wed, Apr 8, 2009 at 9:07 AM, Luke Benstead <kazade(a)gmail.com> wrote:
...
This is probably a really dumb question... but why does wine support UNIX paths? What is the circumstance where a Windows application will be trying to access a native file or directory? The only example I can think of is that an app has specifically been written to be used in Wine, in which case, shouldn't native UNIX paths be disabled by default, and perhaps turned on with an environment variable?
Luke.
I personally enjoy the ability to use UNIX paths both in calling applications with command-line arguments and in file dialogs (even if it is necessary to use the wrong slash in dialogs). I imagine that there are a lot of people that appreciate the ability to use this functionality, since you can use the "familiar paths" you do not need to be familiar with the windows drive mapping. Erich Hoover ehoover(a)mines.edu
On Wed, 8 Apr 2009 09:46:26 -0600 Erich Hoover <ehoover(a)mines.edu> wrote:
On Wed, Apr 8, 2009 at 9:07 AM, Luke Benstead <kazade(a)gmail.com> wrote:
...
This is probably a really dumb question... but why does wine support UNIX paths? What is the circumstance where a Windows application will be trying to access a native file or directory? The only example I can think of is that an app has specifically been written to be used in Wine, in which case, shouldn't native UNIX paths be disabled by default, and perhaps turned on with an environment variable?
Luke.
I personally enjoy the ability to use UNIX paths both in calling applications with command-line arguments and in file dialogs (even if it is necessary to use the wrong slash in dialogs). I imagine that there are a lot of people that appreciate the ability to use this functionality, since you can use the "familiar paths" you do not need to be familiar with the windows drive mapping.
Erich Hoover
I also use wine in somewhat weird way. E.g. I use WinEdt editor under wine to edit TeX files, and it can launch Linux executables like latex, xdvi, dvips and so on. WinEdt takes native Unix paths when it opens files and then passes them to Linux executables. I cannot rely on WinEdt the task to convert Win paths to Unix ones. Thus, if this feature is dropped, I will not be able to use such mixed Win-Lin environments. I believe that Wine should preserve it present behaviour as default and turn on some special treatment of Unix paths (with prefixes etc) on demand, possibly on per-application basis. Oleh Nykyforchyn
2009/4/8 Oleh R. Nykyforchyn <oleh.nyk(a)gmail.com>:
On Wed, 8 Apr 2009 09:46:26 -0600 Erich Hoover <ehoover(a)mines.edu> wrote:
On Wed, Apr 8, 2009 at 9:07 AM, Luke Benstead <kazade(a)gmail.com> wrote:
...
This is probably a really dumb question... but why does wine support UNIX paths? What is the circumstance where a Windows application will be trying to access a native file or directory? The only example I can think of is that an app has specifically been written to be used in Wine, in which case, shouldn't native UNIX paths be disabled by default, and perhaps turned on with an environment variable?
Luke.
I personally enjoy the ability to use UNIX paths both in calling applications with command-line arguments and in file dialogs (even if it is necessary to use the wrong slash in dialogs). I imagine that there are a lot of people that appreciate the ability to use this functionality, since you can use the "familiar paths" you do not need to be familiar with the windows drive mapping.
Erich Hoover
I also use wine in somewhat weird way. E.g. I use WinEdt editor under wine to edit TeX files, and it can launch Linux executables like latex, xdvi, dvips and so on. WinEdt takes native Unix paths when it opens files and then passes them to Linux executables. I cannot rely on WinEdt the task to convert Win paths to Unix ones. Thus, if this feature is dropped, I will not be able to use such mixed Win-Lin environments. I believe that Wine should preserve it present behaviour as default and turn on some special treatment of Unix paths (with prefixes etc) on demand, possibly on per-application basis.
Oleh Nykyforchyn
I disagree. Wine should treat paths *by default* like they are Windows paths. UNIX paths should be identified in the way that Alexandre suggested, with a prefix. That's the best way to make sure that all Windows programs will work as they should, and still allow for UNIX paths. Don't worry, I don't think anyone is suggesting removing UNIX paths completely :) Luke.
Oleh R. Nykyforchyn wrote:
I also use wine in somewhat weird way. E.g. I use WinEdt editor under wine to edit TeX files, and it can launch Linux executables like latex, xdvi, dvips and so on. WinEdt takes native Unix paths when it opens files and then passes them to Linux executables. I cannot rely on WinEdt the task to convert Win paths to Unix ones. Thus, if this feature is dropped, I will not be able to use such mixed Win-Lin environments. I believe that Wine should preserve it present behaviour as default and turn on some special treatment of Unix paths (with prefixes etc) on demand, possibly on per-application basis.
Isn't that why WINE has the Z: drive that maps to / anyway?
On Wed, Apr 8, 2009 at 6:08 PM, Koro <webmaster(a)korosoft.net> wrote:
Isn't that why WINE has the Z: drive that maps to / anyway?
Wine has Z: so that in its default state all Unix paths can map to a Windows path. However, Wine cannot rely on Z: to always map to /. A Wine without Z: is a perfectly valid configuration (although it may mean windows programs cannot access some paths), but paths within C: need to always continue to function. Apparently Wine directly accesses Unix paths internally in some cases that I did not find (I'm not sure how I would go about finding them if I wanted to fix this.), and doing this is preferable (?) to explicitly converting them first. That is, rather than doing: windows_path <- convert_unix_path_to_windows_path(unix_path) open_file(windows_path) we should be doing: open_file(unix_path) Having to add "\\?\unix" to the start of the unix path (so we can do open_file(\\?\unix/unix_path)) is apparently worth skipping the conversion, which still happens internally. I don't know how, but that's what the rejection of the first patch in this series, combined with the \\?\unix idea, tells me. Vincent Povirk
Luke Benstead <kazade(a)gmail.com> writes:
This is probably a really dumb question... but why does wine support UNIX paths? What is the circumstance where a Windows application will be trying to access a native file or directory? The only example I can think of is that an app has specifically been written to be used in Wine, in which case, shouldn't native UNIX paths be disabled by default, and perhaps turned on with an environment variable?
It can be used anywhere an app uses a user-specified path without mangling it too much; admittedly that doesn't happen very often, apps like to mangle paths. There are also places where Wine itself depends on it, to support things like "wine ~/foo.exe" or to allow Unix paths in some registry entries. These are all things that could probably be reimplemented in a more reliable fashion, for instance by using \\?\unix instead of relying on the path detection heuristic. Once this is done properly everywhere, then maybe the hackish way could be removed. -- Alexandre Julliard julliard(a)winehq.org
2009/4/8 Alexandre Julliard <julliard(a)winehq.org>:
Luke Benstead <kazade(a)gmail.com> writes:
This is probably a really dumb question... but why does wine support UNIX paths? What is the circumstance where a Windows application will be trying to access a native file or directory? The only example I can think of is that an app has specifically been written to be used in Wine, in which case, shouldn't native UNIX paths be disabled by default, and perhaps turned on with an environment variable?
It can be used anywhere an app uses a user-specified path without mangling it too much; admittedly that doesn't happen very often, apps like to mangle paths. There are also places where Wine itself depends on it, to support things like "wine ~/foo.exe" or to allow Unix paths in some registry entries.
These are all things that could probably be reimplemented in a more reliable fashion, for instance by using \\?\unix instead of relying on the path detection heuristic. Once this is done properly everywhere, then maybe the hackish way could be removed.
-- Alexandre Julliard julliard(a)winehq.org
That sounds like a far better way of doing it. Perhaps though, if that method was implemented, passing a program path directly into wine (wine ~/foo.exe) would be a special case (without a \\?\ prefix). From a users point of view that's what I'd expect, as you haven't yet started the application (and entered windows land :) ). Luke.
Luke Benstead <kazade(a)gmail.com> writes:
That sounds like a far better way of doing it. Perhaps though, if that method was implemented, passing a program path directly into wine (wine ~/foo.exe) would be a special case (without a \\?\ prefix). From a users point of view that's what I'd expect, as you haven't yet started the application (and entered windows land :) ).
Yes, that's my point, the functionality has to be preserved as far as possible (and not just for the program name). So the heuristic essentially needs to be moved to higher-level functions, which would then in turn call the lower-level code with an explicit \\?\unix prefix. -- Alexandre Julliard julliard(a)winehq.org
participants (15)
-
Alexandre Julliard -
Austin English -
Ben Klein -
Chris Howe -
Chris Robinson -
Dmitry Timoshkov -
Erich Hoover -
Henri Verbeet -
Koro -
Luke Benstead -
Michael Karcher -
Oleh R. Nykyforchyn -
Paul Chitescu -
Peter Rosin -
Vincent Povirk