Fun with argv

[Paul Millar]

Hi everyone,

I've been looking at the installer for Office97. You call setup.exe 
which runs acmsetup.exe with a bunch of options, specifically:
'E:~MSSETUP.T~msstfof.t\acmsetup /T Off97Pro.stf /S "D:\office" '

Here, I've used single quotes to delimit the string. I'll carry on this
convention because it will get confusing otherwise.

The problem is build_argv() assumes the command line is suitably escaped
(as from a bash command line, for example), and so mangles the above line
to the following values:
 'E:~MSSETUP.T~msstfof.t\acmsetup'
 '/T'
 'Off97Pro.stf'
 '/S'
 'D:\office" '

But, the fun continues as you can pass arguments to setup.exe, which 
augment the arguments passed to acmsetup.exe, so running:

wine setup.exe 'a a' '"b "B'

results in argv[] for setup.exe being:
  'setup.exe'
  'a a'
  '"b "B'

setup.exe then calls GetCommandLineA, which returns:
  'setup.exe "a a" ""b "B"'

and then it tries to exec
  'E:~MSSETUP.T~msstfof.t\acmsetup /T Off97Pro.stf /S "D:\office" "a a" 
""b "B"'
A command line with both unix-style escaped quotes and normal text. The 
build_argv result is:
  'E:~MSSETUP.T~msstfof.t\acmsetup'
 [...]
  '/S'
  'D:\office" a'
  'a "b'
  '"B'

and subsequent call to GetCommandLineA returns:
  'E:~MSSETUP.T~msstfof.t\acmsetup /T Off97Pro.stf /S "D:\office" a" "a 
"b" "B'

What a mess!
> From what I can see, the problem is ENV_BuildCommandLine build a command

line that is -escaped (" => " and \ => \). Windows programs might use
this as part of a WinExec call (as above), which in general is not
-escaped.

The only way around this problem (I could think of) is to stop
ENV_BuildCommandLine from -escaping the command line. This implies that
we cannot accept double-quotes on the command line as part of an argument.

I have a patch which is a hack-and-slash to remove the -escaping and the
resulting build processes arguments consistently (and Office97 setup gets
a bit further). However, someone's obviously gone to the trouble of coding
in support for double-quotes, so is this needed? Is there an alternative
solution?

Cheers,

Paul.

[Paul Millar]

Here's the patch, BTW.

ChangeLog:
  Remove support for double quotes within command arguments as standard 
  backslash escaping is incompatible with Windows.

Paul.

On Sat, 16 Feb 2002, Paul Millar wrote:
> Hi everyone,
>
> I've been looking at the installer for Office97. You call setup.exe 
> which runs acmsetup.exe with a bunch of options, specifically:
> 'E:~MSSETUP.T~msstfof.t\acmsetup /T Off97Pro.stf /S "D:\office" '
>
> Here, I've used single quotes to delimit the string. I'll carry on this
> convention because it will get confusing otherwise.
>
> The problem is build_argv() assumes the command line is suitably escaped
> (as from a bash command line, for example), and so mangles the above line
> to the following values:
>  'E:~MSSETUP.T~msstfof.t\acmsetup'
>  '/T'
>  'Off97Pro.stf'
>  '/S'
>  'D:\office" '
>
> But, the fun continues as you can pass arguments to setup.exe, which 
> augment the arguments passed to acmsetup.exe, so running:
>
> wine setup.exe 'a a' '"b "B'
>
> results in argv[] for setup.exe being:
>   'setup.exe'
>   'a a'
>   '"b "B'
>
> setup.exe then calls GetCommandLineA, which returns:
>   'setup.exe "a a" ""b "B"'
>
> and then it tries to exec
>   'E:~MSSETUP.T~msstfof.t\acmsetup /T Off97Pro.stf /S "D:\office" "a a" 
> ""b "B"'
> A command line with both unix-style escaped quotes and normal text. The 
> build_argv result is:
>   'E:~MSSETUP.T~msstfof.t\acmsetup'
>  [...]
>   '/S'
>   'D:\office" a'
>   'a "b'
>   '"B'
>
> and subsequent call to GetCommandLineA returns:
>   'E:~MSSETUP.T~msstfof.t\acmsetup /T Off97Pro.stf /S "D:\office" a" "a 
> "b" "B'
>
> What a mess!
> > From what I can see, the problem is ENV_BuildCommandLine build a command
>
> line that is -escaped (" => " and \ => \). Windows programs might use
> this as part of a WinExec call (as above), which in general is not
> -escaped.
>
> The only way around this problem (I could think of) is to stop
> ENV_BuildCommandLine from -escaping the command line. This implies that
> we cannot accept double-quotes on the command line as part of an argument.
>
> I have a patch which is a hack-and-slash to remove the -escaping and the
> resulting build processes arguments consistently (and Office97 setup gets
> a bit further). However, someone's obviously gone to the trouble of coding
> in support for double-quotes, so is this needed? Is there an alternative
> solution?
>
> Cheers,
>
> Paul.

[Dan Kegel]

Might be nice to have a regression test to check whether
Wine behaves properly with respect to the commandline.

How would one go about writing one?  Would the existing
perl regression test stuff be useful?  Or would a shell
script also be needed (since at least part of the user
stories involve starting wine from the linux commandline,
which isn't something that fits in the portable regression
test scheme imho).
- Dan

Paul Millar wrote:
> Here's the patch, BTW.
>
> ChangeLog:
>   Remove support for double quotes within command arguments as standard
>   backslash escaping is incompatible with Windows.
>
> Paul.
>
> On Sat, 16 Feb 2002, Paul Millar wrote:
> > Hi everyone,
> >
> > I've been looking at the installer for Office97. You call setup.exe
> > which runs acmsetup.exe with a bunch of options, specifically:
> > 'E:~MSSETUP.T~msstfof.t\acmsetup /T Off97Pro.stf /S "D:\office" '
> >
> > Here, I've used single quotes to delimit the string. I'll carry on this
> > convention because it will get confusing otherwise.
> >
> > The problem is build_argv() assumes the command line is suitably escaped
> > (as from a bash command line, for example), and so mangles the above line
> > to the following values:
> >  'E:~MSSETUP.T~msstfof.t\acmsetup'
> >  '/T'
> >  'Off97Pro.stf'
> >  '/S'
> >  'D:\office" '
> >
> > But, the fun continues as you can pass arguments to setup.exe, which
> > augment the arguments passed to acmsetup.exe, so running:
> >
> > wine setup.exe 'a a' '"b "B'
> >
> > results in argv[] for setup.exe being:
> >   'setup.exe'
> >   'a a'
> >   '"b "B'
> >
> > setup.exe then calls GetCommandLineA, which returns:
> >   'setup.exe "a a" ""b "B"'
> >
> > and then it tries to exec
> >   'E:~MSSETUP.T~msstfof.t\acmsetup /T Off97Pro.stf /S "D:\office" "a a"
> > ""b "B"'
> > A command line with both unix-style escaped quotes and normal text. The
> > build_argv result is:
> >   'E:~MSSETUP.T~msstfof.t\acmsetup'
> >  [...]
> >   '/S'
> >   'D:\office" a'
> >   'a "b'
> >   '"B'
> >
> > and subsequent call to GetCommandLineA returns:
> >   'E:~MSSETUP.T~msstfof.t\acmsetup /T Off97Pro.stf /S "D:\office" a" "a
> > "b" "B'
> >
> > What a mess!
> > > From what I can see, the problem is ENV_BuildCommandLine build a command
> >
> > line that is -escaped (" => " and \ => \). Windows programs might use
> > this as part of a WinExec call (as above), which in general is not
> > -escaped.
> >
> > The only way around this problem (I could think of) is to stop
> > ENV_BuildCommandLine from -escaping the command line. This implies that
> > we cannot accept double-quotes on the command line as part of an argument.
> >
> > I have a patch which is a hack-and-slash to remove the -escaping and the
> > resulting build processes arguments consistently (and Office97 setup gets
> > a bit further). However, someone's obviously gone to the trouble of coding
> > in support for double-quotes, so is this needed? Is there an alternative
> > solution?

On Sat, 16 Feb 2002, Paul Millar wrote:
> Hi everyone,
>
> I've been looking at the installer for Office97. You call setup.exe
> which runs acmsetup.exe with a bunch of options, specifically:
> 'E:~MSSETUP.T~msstfof.t\acmsetup /T Off97Pro.stf /S "D:\office" '

Yes, and why did you escapr this trailing quote? >>>>>>>>>>>>>>>>>>^^
> Here, I've used single quotes to delimit the string. I'll carry on this
> convention because it will get confusing otherwise.

It is plenty confusing enough.  What in the name of Babbage are you
trying to do?
> The problem is build_argv() assumes the command line is suitably escaped
> (as from a bash command line, for example), and so mangles the above line
> to the following values:
>  'E:~MSSETUP.T~msstfof.t\acmsetup'
>  '/T'
>  'Off97Pro.stf'
>  '/S'
>  'D:\office" '

No, you mangled it in the command line by excapint the trailing ".
> But, the fun continues as you can pass arguments to setup.exe, which
> augment the arguments passed to acmsetup.exe, so running:
>
> wine setup.exe 'a a' '"b "B'

Now again, why in the name of Babbage do you want to do this?
> results in argv[] for setup.exe being:
>   'setup.exe'
>   'a a'
>   '"b "B'
>
> setup.exe then calls GetCommandLineA, which returns:
>   'setup.exe "a a" ""b "B"'
>
> and then it tries to exec
>   'E:~MSSETUP.T~msstfof.t\acmsetup /T Off97Pro.stf /S "D:\office" "a a"

I didn't do the command line quoting, but I've tested it some, and I
think it is essentailly correct, given what the C runtime routine is
going to do to split out argv[] again.  What is it you are trying to do?
Okay, maybe it is not exactly intuitive that "D:\office" is a directory
name (the \ is not an escape) but "D:\office" is wrong.  If you want
that trailing \ to come out in the argv, (after the C runtime has had
its way with it) you need to escape the , not the ",
so:  "D:\office\".

Lawson
---oof---

[Paul Millar]

On Sat, 16 Feb 2002 [e-mail] wrote:
> On Sat, 16 Feb 2002, Paul Millar wrote:
> > I've been looking at the installer for Office97. You call setup.exe
> > which runs acmsetup.exe with a bunch of options, specifically:
> > 'E:~MSSETUP.T~msstfof.t\acmsetup /T Off97Pro.stf /S "D:\office" '
>
> Yes, and why did you escapr this trailing quote? >>>>>>>>>>>>>>>>>>^^

I didn't. (and it isn't escaped: think Windows rather than *nix)

This command line is _from_ MS's setup.exe calling another program
acmsetup.exe. I didn't choose to have the quote marks around the
D:\office\ bit. Setup.exe doesn't do _any_ escaping. It's one of those
unfortunate things that, in this case, a non-escaped string looks like an
escaped one. If it didn't, we'd get away with the current command line
parsing.
> > Here, I've used single quotes to delimit the string. I'll carry on this
> > convention because it will get confusing otherwise.
>
> It is plenty confusing enough.  What in the name of Babbage are you
> trying to do?

Trying (and succeeding, to some extent ;) to get setup.exe working.

With the single quotes stuff, I was trying to show that there's a space at
the end of the command line. This turns out to be irrelevent, but I
thought it worth showing it.

If its too confusing, do
  sed -e "s/'//g" < email
and be done with them :^)
> > The problem is build_argv() assumes the command line is suitably escaped
> > (as from a bash command line, for example), and so mangles the above line
> > to the following values:
> >  'E:~MSSETUP.T~msstfof.t\acmsetup'
> >  '/T'
> >  'Off97Pro.stf'
> >  '/S'
> >  'D:\office" '
>
> No, you mangled it in the command line by excapint the trailing ".

Nope. Setup.exe passed the command line with something that _looks_ like
its escaped. Wine's current behavour assumes command lines are escaped and
acts accordingly, thus it mangled the command line.
> > But, the fun continues as you can pass arguments to setup.exe, which
> > augment the arguments passed to acmsetup.exe, so running:
> >
> > wine setup.exe 'a a' '"b "B'
>
> Now again, why in the name of Babbage do you want to do this?

Its an awkward command line. I think it illustrates the problem with
escaping double quotes, because everything gets messed up. With the fix,
everything behaves "correctly", and the two extra arguments 'a a' and
'b B' are passed to acmsetup.exe correctly (without the offending double 
quotes).

[snip!]
> I didn't do the command line quoting, but I've tested it some, and I
> think it is essentailly correct, given what the C runtime routine is
> going to do to split out argv[] again.  What is it you are trying to do?

I'm trying to fix an inconsistency. The problem is we're currently mixing
*nix-style command line (which _require_ double quotes to be escaped) with
Windows-style command lines (which have _no_ escaping). The only way you
can have the two mixed together is if you remove the possibility of having
double-quotes in an argument.
> Okay, maybe it is not exactly intuitive that "D:\office" is a directory
> name (the \ is not an escape) but "D:\office" is wrong.

'fraid it can't be wrong. Its what setup.exe does and it works under 
Windows. If we want to support everything in Windows, we have to support 
this too.

Also, (please) correct me if I'm wrong, but AFAIK there is no way of
putting an escaped double quote into an argument within Windows. It
doesn't do escaping, so any " is treated as the start of a quoted region.

Because of this "blah" is perfectly valid in Windows, but wrong on a *nix 
cmd line.
> If you want
> that trailing \ to come out in the argv, (after the C runtime has had
> its way with it) you need to escape the , not the ",
> so:  "D:\office\".

Well, more "D:\office\", but unfortunately Windows (or at least 
setup.exe) doesn't do it that way.

Cheers,

Paul.

On Sun, 17 Feb 2002, Paul Millar wrote:
> I didn't. (and it isn't escaped: think Windows rather than *nix)

Sorry, I can't.  Wine is as close to Windows as I ever got.
I'll think GCOS-8 or MVS if you like, but I don't think it will help.
> This command line is _from_ MS's setup.exe calling another program
> acmsetup.exe. I didn't choose to have the quote marks around the
> D:\office\ bit. Setup.exe doesn't do _any_ escaping. It's one of those
> unfortunate things that, in this case, a non-escaped string looks like an
> escaped one. If it didn't, we'd get away with the current command line
> parsing.

Okay, that makes a difference.  But you did put in those funky 'a a' and
'"b "B' things, no?
> Trying (and succeeding, to some extent ;) to get setup.exe working.
>
> With the single quotes stuff, I was trying to show that there's a space at
> the end of the command line. This turns out to be irrelevent, but I
> thought it worth showing it.
>
> If its too confusing, do
>   sed -e "s/'//g" < email
> and be done with them :^)

Singlle quotes are very useful and not awfully confusing.  What is
confusing is the Windows "cammand line" 8-D.
> Nope. Setup.exe passed the command line with something that _looks_ like
> its escaped. Wine's current behavour assumes command lines are escaped and
> acts accordingly, thus it mangled the command line.

Okay.  How would it be if we provided for a trailing backslash to not be
interpreted as an escape?  I am afraid if we rip out all escaping and
interpretation of escapes, we are going to make Wine more difficult to
use as a *nix program, which of coourse it is.
> Its an awkward command line. I think it illustrates the problem with
> escaping double quotes, because everything gets messed up. With the fix,
> everything behaves "correctly", and the two extra arguments 'a a' and
> 'b B' are passed to acmsetup.exe correctly (without the offending double
> quotes).

The code here doesn't normally see quotes, single or double, after the
shell is done, but intuits that they were there if it sees spaces in a
*NIX argv string, so it puts them back when pasting the line back
together.
> I'm trying to fix an inconsistency. The problem is we're currently mixing
> *nix-style command line (which _require_ double quotes to be escaped) with
> Windows-style command lines (which have _no_ escaping). The only way you
> can have the two mixed together is if you remove the possibility of having
> double-quotes in an argument.
>
> 'fraid it can't be wrong. Its what setup.exe does and it works under
> Windows. If we want to support everything in Windows, we have to support
> this too.

Wine is already better than Windows.  Now we just have to get a few
obscure features to work.
> Also, (please) correct me if I'm wrong, but AFAIK there is no way of
> putting an escaped double quote into an argument within Windows. It
> doesn't do escaping, so any " is treated as the start of a quoted region.
>
> Because of this "blah" is perfectly valid in Windows, but wrong on a *nix
> cmd line.

That's what I was afraid of.  \ is handled correctly unless it comes
before " at the end of a string, I think.
> Well, more "D:\office\", but unfortunately Windows (or at least
> setup.exe) doesn't do it that way.

The whole thing was inside '', so you didn't need to escape , unless it
comes before " or so, I think.
> Cheers,
>
> Paul.

Lawson

For Windows to triumph it is only necessary that good men do nothing.

[Francois Gouget]

It seems commandline/argv issues keep coming up. Well, I have
nothing to add since last time. Have a look at the following
mailing-list threads:

* 2002-01-10: Re: Handle quotes in command lines
   http://www.winehq.com/hypermail/wine-devel/2002/01/0215.html
   and especially suggestions for fixing this:
   http://www.winehq.com/hypermail/wine-devel/2002/01/0240.html

* 2001-09-02: ' ' & '"' handling in the command line
   http://www.winehq.com/hypermail/wine-devel/2001/09/0004.html

--
Francois Gouget         [e-mail]        http://fgouget.free.fr/
In theory, theory and practice are the same, but in practice they're different.