oleaut32: ITypeInfo_fnInvoke behavior on PARAMFLAG_FOPT without PARAMFLAG_FHASDEFAULT

In a tax app written in Visual Basic/ADO, I found the following problem:

In dlls/oleaut32/typelib.c, in function ITypeInfo_fnInvoke(), the
following code is found (around line 5569).

VARIANTARG *missing_arg = INVBUF_GET_MISSING_ARG_ARRAY(buffer,
func_desc->cParams);

V_VT(arg) = VT_VARIANT | VT_BYREF;
V_VARIANTREF(arg) = &missing_arg[i];
V_VT(V_VARIANTREF(arg)) = VT_ERROR;
V_ERROR(V_VARIANTREF(arg)) = DISP_E_PARAMNOTFOUND;

This code runs when a parameter is found that has the PARAMFLAG_FIN and
PARAMFLAG_FOPT flags, but not the PARAMFLAG_FHASDEFAULT flag. As far as I
understand, this code is supposed to supply a variant that might hold an
output value. However, in the Append method of the Fields object within
the Recordset object of Microsoft ADODB 2.7, the following declaration is
found:

trace:ole:ITypeInfo_fnInvoke invoking:
L"Append"(5)
    parm0: L"Name"
    parm1: L"Type"
    parm2: L"DefinedSize"
    parm3: L"Attrib"
    parm4: L"FieldValue"
memid is 00000003
Param 0:
    	tdesc.vartype 8 (VT_BSTR)
    	u.paramdesc.wParamFlags PARAMFLAG_FIN
    	u.paramdesc.lpex (nil)
Param 1:
    	tdesc.vartype 29 (VT_USERDEFINED ref = 2bc)
    	u.paramdesc.wParamFlags PARAMFLAG_FIN
    	u.paramdesc.lpex (nil)
Param 2:
    	tdesc.vartype 29 (VT_USERDEFINED ref = 1c84)
    	u.paramdesc.wParamFlags PARAMFLAG_FIN PARAMFLAG_FOPT PARAMFLAG_FHASDEFAULT
    	u.paramdesc.lpex 0x199868
Param 3:
    	tdesc.vartype 29 (VT_USERDEFINED ref = 320)
    	u.paramdesc.wParamFlags PARAMFLAG_FIN PARAMFLAG_FOPT PARAMFLAG_FHASDEFAULT
    	u.paramdesc.lpex 0x1998d0
Param 4:
    	tdesc.vartype 12 (VT_VARIANT)
    	u.paramdesc.wParamFlags PARAMFLAG_FIN PARAMFLAG_FOPT
    	u.paramdesc.lpex (nil)
    funckind: 1 (pure virtual)
    invkind: 1 (func)
    callconv: 4 (stdcall)
    oVft: 52
    cParamsOpt: 1
    wFlags: 0
    elemdescFunc (return value type):
    	tdesc.vartype 25 (VT_HRESULT)
    	u.paramdesc.wParamFlags PARAMFLAGS_NONE
    	u.paramdesc.lpex (nil)
    helpstring: (null)
    entry (ordinal): 0x0000

The implementation of the method in Microsoft ADODB 2.7 last parameter
(FieldValue) absolutely refuses to accept a VT_VARIANT | VT_BYREF variant.
This result is not dependent of the type of variant pointed to by the
first-level variant - changing VT_ERROR to VT_EMPTY has no effect. Only by
passing a VT_EMPTY instead of a VT_VARIANT | VT_BYREF can the application
proceed normally. All oleaut32 tests still pass. Please comment if this is
an appropriate fix for this issue.

Changelog:
* (PARAMFLAG_FIN | PARAMFLAG_FOPT) & ~PARAMFLAG_FHASDEFAULT should result
in a default parameter of VT_EMPTY, not a VT_VARIANT | VT_BYREF in
ITypeInfo_fnInvoke

Alex Villacís Lasso

[Robert Shearman]

[e-mail] wrote:
> In a tax app written in Visual Basic/ADO, I found the following problem:
>
> In dlls/oleaut32/typelib.c, in function ITypeInfo_fnInvoke(), the
> following code is found (around line 5569).
>
> VARIANTARG *missing_arg = INVBUF_GET_MISSING_ARG_ARRAY(buffer,
> func_desc->cParams);
>
> V_VT(arg) = VT_VARIANT | VT_BYREF;
> V_VARIANTREF(arg) = &missing_arg[i];
> V_VT(V_VARIANTREF(arg)) = VT_ERROR;
> V_ERROR(V_VARIANTREF(arg)) = DISP_E_PARAMNOTFOUND;
>
> This code runs when a parameter is found that has the PARAMFLAG_FIN and
> PARAMFLAG_FOPT flags, but not the PARAMFLAG_FHASDEFAULT flag. As far as I
> understand, this code is supposed to supply a variant that might hold an
> output value. However, in the Append method of the Fields object within
> the Recordset object of Microsoft ADODB 2.7, the following declaration is
> found:
>
> trace:ole:ITypeInfo_fnInvoke invoking:
> L"Append"(5)
>    parm0: L"Name"
>    parm1: L"Type"
>    parm2: L"DefinedSize"
>    parm3: L"Attrib"
>    parm4: L"FieldValue"
> memid is 00000003
> Param 0:
>    	tdesc.vartype 8 (VT_BSTR)
>    	u.paramdesc.wParamFlags PARAMFLAG_FIN
>    	u.paramdesc.lpex (nil)
> Param 1:
>    	tdesc.vartype 29 (VT_USERDEFINED ref = 2bc)
>    	u.paramdesc.wParamFlags PARAMFLAG_FIN
>    	u.paramdesc.lpex (nil)
> Param 2:
>    	tdesc.vartype 29 (VT_USERDEFINED ref = 1c84)
>    	u.paramdesc.wParamFlags PARAMFLAG_FIN PARAMFLAG_FOPT PARAMFLAG_FHASDEFAULT
>    	u.paramdesc.lpex 0x199868
> Param 3:
>    	tdesc.vartype 29 (VT_USERDEFINED ref = 320)
>    	u.paramdesc.wParamFlags PARAMFLAG_FIN PARAMFLAG_FOPT PARAMFLAG_FHASDEFAULT
>    	u.paramdesc.lpex 0x1998d0
> Param 4:
>    	tdesc.vartype 12 (VT_VARIANT)
>    	u.paramdesc.wParamFlags PARAMFLAG_FIN PARAMFLAG_FOPT
>    	u.paramdesc.lpex (nil)
>    funckind: 1 (pure virtual)
>    invkind: 1 (func)
>    callconv: 4 (stdcall)
>    oVft: 52
>    cParamsOpt: 1
>    wFlags: 0
>    elemdescFunc (return value type):
>    	tdesc.vartype 25 (VT_HRESULT)
>    	u.paramdesc.wParamFlags PARAMFLAGS_NONE
>    	u.paramdesc.lpex (nil)
>    helpstring: (null)
>    entry (ordinal): 0x0000
>
> The implementation of the method in Microsoft ADODB 2.7 last parameter
> (FieldValue) absolutely refuses to accept a VT_VARIANT | VT_BYREF variant.
> This result is not dependent of the type of variant pointed to by the
> first-level variant - changing VT_ERROR to VT_EMPTY has no effect. Only by
> passing a VT_EMPTY instead of a VT_VARIANT | VT_BYREF can the application
> proceed normally. All oleaut32 tests still pass. Please comment if this is
> an appropriate fix for this issue.
>
> Changelog:
> - (PARAMFLAG_FIN | PARAMFLAG_FOPT) & ~PARAMFLAG_FHASDEFAULT should result
>
> in a default parameter of VT_EMPTY, not a VT_VARIANT | VT_BYREF in
> ITypeInfo_fnInvoke

I don't believe this is fix is correct. See the tests not included in 
Wine yet here: 
http://www.winehq.org/pipermail/wine-patches/2006-August/029511.html

In particular, this code:
> +HRESULT WINAPI Widget_DoSomething(
> - IWidget __RPC_FAR * iface,
> - /* [in] */ double number,
> - /* [out] */ BSTR *str1,
> - /* [defaultvalue][in] */ BSTR str2,
> - /* [optional][in] */ VARIANT __RPC_FAR *opt)
>
> +{
> - static const WCHAR szString[] = { 'S','t','r','i','n','g',0 };
> - trace("DoSomething()\n");
> -
> - ok(number == 3.141, "number(%f) != 3.141\n", number);
> - ok(*str2 == '\0', "str2(%s) != ""\n", wine_dbgstr_w(str2));
> - ok(V_VT(opt) == VT_ERROR, "V_VT(opt) should be VT_ERROR instead
>
> of 0x%x\n", V_VT(opt));
> - ok(V_ERROR(opt) == DISP_E_PARAMNOTFOUND, "V_ERROR(opt) should be
>
> DISP_E_PARAMNOTFOUND instead of 0x%08lx\n", V_ERROR(opt));
> - *str1 = SysAllocString(szString);
> -
> - return S_FALSE;
>
> +}

[Alex Villací­s Lasso]

Robert Shearman escribió:
> [e-mail] wrote:
> > In a tax app written in Visual Basic/ADO, I found the following problem:
> >
> > In dlls/oleaut32/typelib.c, in function ITypeInfo_fnInvoke(), the
> > following code is found (around line 5569).
> >
> > VARIANTARG *missing_arg = INVBUF_GET_MISSING_ARG_ARRAY(buffer,
> > func_desc->cParams);
> >
> > V_VT(arg) = VT_VARIANT | VT_BYREF;
> > V_VARIANTREF(arg) = &missing_arg[i];
> > V_VT(V_VARIANTREF(arg)) = VT_ERROR;
> > V_ERROR(V_VARIANTREF(arg)) = DISP_E_PARAMNOTFOUND;
> >
> > This code runs when a parameter is found that has the PARAMFLAG_FIN and
> > PARAMFLAG_FOPT flags, but not the PARAMFLAG_FHASDEFAULT flag. As far 
> > as I
> > understand, this code is supposed to supply a variant that might hold an
> > output value. However, in the Append method of the Fields object within
> > the Recordset object of Microsoft ADODB 2.7, the following 
> > declaration is
> > found:
> >
> > trace:ole:ITypeInfo_fnInvoke invoking:
> > L"Append"(5)
> >     parm0: L"Name"
> >     parm1: L"Type"
> >     parm2: L"DefinedSize"
> >     parm3: L"Attrib"
> >     parm4: L"FieldValue"
> > memid is 00000003
> > Param 0:
> >         tdesc.vartype 8 (VT_BSTR)
> >         u.paramdesc.wParamFlags PARAMFLAG_FIN
> >         u.paramdesc.lpex (nil)
> > Param 1:
> >         tdesc.vartype 29 (VT_USERDEFINED ref = 2bc)
> >         u.paramdesc.wParamFlags PARAMFLAG_FIN
> >         u.paramdesc.lpex (nil)
> > Param 2:
> >         tdesc.vartype 29 (VT_USERDEFINED ref = 1c84)
> >         u.paramdesc.wParamFlags PARAMFLAG_FIN PARAMFLAG_FOPT 
> > PARAMFLAG_FHASDEFAULT
> >         u.paramdesc.lpex 0x199868
> > Param 3:
> >         tdesc.vartype 29 (VT_USERDEFINED ref = 320)
> >         u.paramdesc.wParamFlags PARAMFLAG_FIN PARAMFLAG_FOPT 
> > PARAMFLAG_FHASDEFAULT
> >         u.paramdesc.lpex 0x1998d0
> > Param 4:
> >         tdesc.vartype 12 (VT_VARIANT)
> >         u.paramdesc.wParamFlags PARAMFLAG_FIN PARAMFLAG_FOPT
> >         u.paramdesc.lpex (nil)
> >     funckind: 1 (pure virtual)
> >     invkind: 1 (func)
> >     callconv: 4 (stdcall)
> >     oVft: 52
> >     cParamsOpt: 1
> >     wFlags: 0
> >     elemdescFunc (return value type):
> >         tdesc.vartype 25 (VT_HRESULT)
> >         u.paramdesc.wParamFlags PARAMFLAGS_NONE
> >         u.paramdesc.lpex (nil)
> >     helpstring: (null)
> >     entry (ordinal): 0x0000
> >
> > The implementation of the method in Microsoft ADODB 2.7 last parameter
> > (FieldValue) absolutely refuses to accept a VT_VARIANT | VT_BYREF 
> > variant.
> > This result is not dependent of the type of variant pointed to by the
> > first-level variant - changing VT_ERROR to VT_EMPTY has no effect. 
> > Only by
> > passing a VT_EMPTY instead of a VT_VARIANT | VT_BYREF can the 
> > application
> > proceed normally. All oleaut32 tests still pass. Please comment if 
> > this is
> > an appropriate fix for this issue.
> >
> > Changelog:
> > - (PARAMFLAG_FIN | PARAMFLAG_FOPT) & ~PARAMFLAG_FHASDEFAULT should
> >
> > result
> > in a default parameter of VT_EMPTY, not a VT_VARIANT | VT_BYREF in
> > ITypeInfo_fnInvoke
>
> I don't believe this is fix is correct. See the tests not included in 
> Wine yet here: 
> http://www.winehq.org/pipermail/wine-patches/2006-August/029511.html
>
> In particular, this code:
> > +HRESULT WINAPI Widget_DoSomething(
> > - IWidget __RPC_FAR * iface,
> > - /* [in] */ double number,
> > - /* [out] */ BSTR *str1,
> > - /* [defaultvalue][in] */ BSTR str2,
> > - /* [optional][in] */ VARIANT __RPC_FAR *opt)
> >
> > +{
> > - static const WCHAR szString[] = { 'S','t','r','i','n','g',0 };
> > - trace("DoSomething()\n");
> > -
> > - ok(number == 3.141, "number(%f) != 3.141\n", number);
> > - ok(*str2 == '\0', "str2(%s) != ""\n", wine_dbgstr_w(str2));
> > - ok(V_VT(opt) == VT_ERROR, "V_VT(opt) should be VT_ERROR instead
> >
> > of 0x%x\n", V_VT(opt));
> > - ok(V_ERROR(opt) == DISP_E_PARAMNOTFOUND, "V_ERROR(opt) should be
> >
> > DISP_E_PARAMNOTFOUND instead of 0x%08lx\n", V_ERROR(opt));
> > - *str1 = SysAllocString(szString);
> > -
> > - return S_FALSE;
> >
> > +}

Bug http://bugs.winehq.org/show_bug.cgi?id=6638 was opened for this 
problem. I have reasons to believe that this is a case of 
(undocumented?) overloading with builtin oleaut32 calling the wrong 
version of a method. The bug report has a full explanation of this, and 
the reasons why I arrived at the overloading theory. However I need an 
OLE Automation expert to verify my hypothesis. Please comment on this 
(especially from the Codeweavers people).

oleaut32: implement underscore-prefix overload (was Re: oleaut32: ITypeInfo_fnInvoke behavior on PARAMFLAG_FOPT without PARAMFLAG_FHASDEFAULT)

[Alex Villací­s Lasso]

Alex Villací­s Lasso escribió:
> Robert Shearman escribió:
> > [e-mail] wrote:
> > > In a tax app written in Visual Basic/ADO, I found the following 
> > > problem:
> > >
> > > In dlls/oleaut32/typelib.c, in function ITypeInfo_fnInvoke(), the
> > > following code is found (around line 5569).
> > >
> > > VARIANTARG *missing_arg = INVBUF_GET_MISSING_ARG_ARRAY(buffer,
> > > func_desc->cParams);
> > >
> > > V_VT(arg) = VT_VARIANT | VT_BYREF;
> > > V_VARIANTREF(arg) = &missing_arg[i];
> > > V_VT(V_VARIANTREF(arg)) = VT_ERROR;
> > > V_ERROR(V_VARIANTREF(arg)) = DISP_E_PARAMNOTFOUND;
> > >
> > > This code runs when a parameter is found that has the PARAMFLAG_FIN and
> > > PARAMFLAG_FOPT flags, but not the PARAMFLAG_FHASDEFAULT flag. As far 
> > > as I
> > > understand, this code is supposed to supply a variant that might 
> > > hold an
> > > output value. However, in the Append method of the Fields object within
> > > the Recordset object of Microsoft ADODB 2.7, the following 
> > > declaration is
> > > found:
> > >
> > > trace:ole:ITypeInfo_fnInvoke invoking:
> > > L"Append"(5)
> > >     parm0: L"Name"
> > >     parm1: L"Type"
> > >     parm2: L"DefinedSize"
> > >     parm3: L"Attrib"
> > >     parm4: L"FieldValue"
> > > memid is 00000003
> > > Param 0:
> > >         tdesc.vartype 8 (VT_BSTR)
> > >         u.paramdesc.wParamFlags PARAMFLAG_FIN
> > >         u.paramdesc.lpex (nil)
> > > Param 1:
> > >         tdesc.vartype 29 (VT_USERDEFINED ref = 2bc)
> > >         u.paramdesc.wParamFlags PARAMFLAG_FIN
> > >         u.paramdesc.lpex (nil)
> > > Param 2:
> > >         tdesc.vartype 29 (VT_USERDEFINED ref = 1c84)
> > >         u.paramdesc.wParamFlags PARAMFLAG_FIN PARAMFLAG_FOPT 
> > > PARAMFLAG_FHASDEFAULT
> > >         u.paramdesc.lpex 0x199868
> > > Param 3:
> > >         tdesc.vartype 29 (VT_USERDEFINED ref = 320)
> > >         u.paramdesc.wParamFlags PARAMFLAG_FIN PARAMFLAG_FOPT 
> > > PARAMFLAG_FHASDEFAULT
> > >         u.paramdesc.lpex 0x1998d0
> > > Param 4:
> > >         tdesc.vartype 12 (VT_VARIANT)
> > >         u.paramdesc.wParamFlags PARAMFLAG_FIN PARAMFLAG_FOPT
> > >         u.paramdesc.lpex (nil)
> > >     funckind: 1 (pure virtual)
> > >     invkind: 1 (func)
> > >     callconv: 4 (stdcall)
> > >     oVft: 52
> > >     cParamsOpt: 1
> > >     wFlags: 0
> > >     elemdescFunc (return value type):
> > >         tdesc.vartype 25 (VT_HRESULT)
> > >         u.paramdesc.wParamFlags PARAMFLAGS_NONE
> > >         u.paramdesc.lpex (nil)
> > >     helpstring: (null)
> > >     entry (ordinal): 0x0000
> > >
> > > The implementation of the method in Microsoft ADODB 2.7 last parameter
> > > (FieldValue) absolutely refuses to accept a VT_VARIANT | VT_BYREF 
> > > variant.
> > > This result is not dependent of the type of variant pointed to by the
> > > first-level variant - changing VT_ERROR to VT_EMPTY has no effect. 
> > > Only by
> > > passing a VT_EMPTY instead of a VT_VARIANT | VT_BYREF can the 
> > > application
> > > proceed normally. All oleaut32 tests still pass. Please comment if 
> > > this is
> > > an appropriate fix for this issue.
> > >
> > > Changelog:
> > > - (PARAMFLAG_FIN | PARAMFLAG_FOPT) & ~PARAMFLAG_FHASDEFAULT should
> > >
> > > result
> > > in a default parameter of VT_EMPTY, not a VT_VARIANT | VT_BYREF in
> > > ITypeInfo_fnInvoke
> >
> > I don't believe this is fix is correct. See the tests not included in 
> > Wine yet here: 
> > http://www.winehq.org/pipermail/wine-patches/2006-August/029511.html
> >
> > In particular, this code:
> > > +HRESULT WINAPI Widget_DoSomething(
> > > - IWidget __RPC_FAR * iface,
> > > - /* [in] */ double number,
> > > - /* [out] */ BSTR *str1,
> > > - /* [defaultvalue][in] */ BSTR str2,
> > > - /* [optional][in] */ VARIANT __RPC_FAR *opt)
> > >
> > > +{
> > > - static const WCHAR szString[] = { 'S','t','r','i','n','g',0 };
> > > - trace("DoSomething()\n");
> > > -
> > > - ok(number == 3.141, "number(%f) != 3.141\n", number);
> > > - ok(*str2 == '\0', "str2(%s) != ""\n", wine_dbgstr_w(str2));
> > > - ok(V_VT(opt) == VT_ERROR, "V_VT(opt) should be VT_ERROR instead
> > >
> > > of 0x%x\n", V_VT(opt));
> > > - ok(V_ERROR(opt) == DISP_E_PARAMNOTFOUND, "V_ERROR(opt) should
> > >
> > > be DISP_E_PARAMNOTFOUND instead of 0x%08lx\n", V_ERROR(opt));
> > > - *str1 = SysAllocString(szString);
> > > -
> > > - return S_FALSE;
> > >
> > > +}
>
> Bug http://bugs.winehq.org/show_bug.cgi?id=6638 was opened for this 
> problem. I have reasons to believe that this is a case of 
> (undocumented?) overloading with builtin oleaut32 calling the wrong 
> version of a method. The bug report has a full explanation of this, 
> and the reasons why I arrived at the overloading theory. However I 
> need an OLE Automation expert to verify my hypothesis. Please comment 
> on this (especially from the Codeweavers people).

Here is a patch that implements my hunch. This patch fixes the problem 
on both my test app at bug #6638 and the tax app I try to run.

Changelog:
* When there are not enough actual parameters for invoking, look for a 
possible overload with underscore prefix before trying the original 
function.

oleaut32: implement underscore-prefix overload (was Re: oleaut32: ITypeInfo_fnInvoke behavior on PARAMFLAG_FOPT without PARAMFLAG_FHASDEFAULT)

[Robert Shearman]

Alex Villací­s Lasso wrote:
> Here is a patch that implements my hunch. This patch fixes the problem 
> on both my test app at bug #6638 and the tax app I try to run.

There are a lot of wrong ways to fix the particular bug you are looking 
at, but only one right way. If you have a doubt, then you should write a 
test program.

Anyway, does the attached patch fix the test app for you?

oleaut32: implement underscore-prefix overload (was Re: oleaut32: ITypeInfo_fnInvoke behavior on PARAMFLAG_FOPT without PARAMFLAG_FHASDEFAULT)

[Alex Villací­s Lasso]

Robert Shearman escribió:
> Alex Villací­s Lasso wrote:
> > Here is a patch that implements my hunch. This patch fixes the 
> > problem on both my test app at bug #6638 and the tax app I try to run.
>
> There are a lot of wrong ways to fix the particular bug you are 
> looking at, but only one right way. If you have a doubt, then you 
> should write a test program.
>
> Anyway, does the attached patch fix the test app for you?
>
> diff --git a/dlls/oleaut32/typelib.c b/dlls/oleaut32/typelib.c
> index a12deb1..263d2c2 100644
> --- a/dlls/oleaut32/typelib.c
> +++ b/dlls/oleaut32/typelib.c
> @@ -5562,11 +5562,15 @@ static HRESULT WINAPI ITypeInfo_fnInvoke
>                      }
>                      else
>                      {
> -
>                    VARIANTARG *missing_arg = INVBUF_GET_MISSING_ARG_ARRAY(buffer, func_desc->cParams);
> -
>                    V_VT(arg) = VT_VARIANT | VT_BYREF;
> -
>                    V_VARIANTREF(arg) = &missing_arg[i];
> -
>                    V_VT(V_VARIANTREF(arg)) = VT_ERROR;
> -
>                    V_ERROR(V_VARIANTREF(arg)) = DISP_E_PARAMNOTFOUND;
> -
>                    if (rgvt[i] == (VT_VARIANT | VT_BYREF))
> -
>                    {
> -
>                        VARIANTARG *missing_arg = INVBUF_GET_MISSING_ARG_ARRAY(buffer, func_desc->cParams);
> -
>                        V_VT(arg) = VT_VARIANT | VT_BYREF;
> -
>                        V_VARIANTREF(arg) = &missing_arg[i];
> -
>                        arg = &missing_arg[i];
> -
>                    }
> -
>                    V_VT(arg) = VT_ERROR;
> -
>                    V_ERROR(arg) = DISP_E_PARAMNOTFOUND;
>                }
>            }
>            else

Yes, it does fix the bug. Thanks! So, was it only a coincidence that the 
failing case was an Append procedure in a class that also had an _Append 
procedure?