[Bug 14639] New: ngen.exe from .NET 2.0 show exception E_INVALIDARG when trying to compile an assembly

http://bugs.winehq.org/show_bug.cgi?id=14639

Summary: ngen.exe from .NET 2.0 show exception E_INVALIDARG when
                    trying to compile an assembly
           Product: Wine
           Version: 1.1.2
          Platform: Other
        OS/Version: other
            Status: NEW
          Keywords: dotnet
          Severity: normal
          Priority: P2
         Component: oleaut32
        AssignedTo: [e-mail]
        ReportedBy: [e-mail]

Created an attachment (id=15047)
 --> (http://bugs.winehq.org/attachment.cgi?id=15047)
oleaut32 patch

To reproduce:

wine ngen asiduvaoisduvbaios

The expected behaviour is HRESULT 0x80070002 (file not found), while we get
E_INVALIDARG. The E_INVALIDARD is because of some garbage provided to
SetPriorityClass.

There is a bug in either oleaut32's serialize_param or deserialize_param -
serialize_param treats VT_USERDEFINED/TKIND_RECORD as inline data, while
deserialize_param makes of it a pointer to an allocated buffer. As the VT_PTR
is absent, I would except the first is correct (also it seems this is what ngen
is expecting). The attached patch makes the class sent to SetPriorityClass
constant - always 0xffffffff.

This is still incorrect. The native SetPriotityClass ignores incorrect flags,
so maybe we should just return TRUE, but it could be another bug e.g. in
oleaut32.

[Bug 14639] ngen.exe from .NET 2.0 show exception E_INVALIDARG when trying to compile an assembly

http://bugs.winehq.org/show_bug.cgi?id=14639

Anastasius Focht  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #1 from Anastasius Focht   2008-07-26 10:42:41 ---
Hello,

the behaviour you are observing (e.g. SetPriorityClass() call with priority
class param = -1) is a result of earlier damage.
To be honest, the real problem is not visible using (relay/olerelay) trace
output, one has to debug through the mess.

It's a COM proxy/server call which imbalances the stack, resulting in funny
behaviour ... 
Pure coincidence due to a combination of compiler generated EH prolog/epilog
code prevents a page fault and actually disguises the location.
The real problem is that wine doesn't handle VT_USERDEFINED args in typelib
marshaler properly.

Consider the following typelib info snippet:

--- snip ---
..

typedef struct tag_NGenPrivateAttributes {
unsigned long Flags;
unsigned long ZapStats;
BSTR DbgDir;
} _NGenPrivateAttributes;

interface ICorSvcSetPrivateAttributes : IUnknown {
    HRESULT _stdcall SetNGenPrivateAttributes([in] _NGenPrivateAttributes
ngenPrivateAttributes);
};

..
--- snip ---

When the proxy entry point code for SetNGenPrivateAttributes() method is setup,
the size of argument stack is calculated by _argsize().
Because of fall through, VT_USERDEFINED argument size will be ever one DWORD,
which is incorrect.

The custom type (struct) is passed by value.
In this example, 3 DWORDs are put on stack (0,0,NULL).
Wine has to take care of this, recursively walking through type descriptors for
VT_USERDEFINED args, calculating the sizes of the structure fields so the stack
can be properly cleaned up after returning from proxy.

I hope I gave enough information fix this...

Regards

[Bug 14639] ngen.exe from .NET 2.0 show exception E_INVALIDARG when trying to compile an assembly

http://bugs.winehq.org/show_bug.cgi?id=14639

Mikolaj Zalewski  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #15047|0                           |1
        is obsolete|                            |

--- Comment #2 from Mikolaj Zalewski   2008-07-31 15:26:10 ---
Created an attachment (id=15192)
 --> (http://bugs.winehq.org/attachment.cgi?id=15192)
patch

This is a better implementation of VT_USERDEFINED parameters, that changes also
_argsize. With it this ngen returns HRESULT 80070002. I will update also _xsize
(to support arrays of structures), test it a bit more, write a test for it and
send it to wine-patches.

[Bug 14639] ngen.exe from .NET 2.0 show exception E_INVALIDARG when trying to compile an assembly

http://bugs.winehq.org/show_bug.cgi?id=14639

Mikolaj Zalewski  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #3 from Mikolaj Zalewski   2008-09-16 09:16:24 ---
Fixed by commits 8b36e71590690330d24831472e072d3ef46b4249,
cc86f6890fd01f6843d7a7b5ecce3ed2f0b08857 and
5c2fd1b192380fb741781db55b8fb93c2a336810 . The fix will be included in Wine
development releases starting from Wine 1.1.5. It might get included in the
next stable release 1.0.1.

[Bug 14639] ngen.exe from .NET 2.0 show exception E_INVALIDARG when trying to compile an assembly

http://bugs.winehq.org/show_bug.cgi?id=14639

Alexandre Julliard  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #4 from Alexandre Julliard   2008-09-19 11:17:39 ---
Closing bugs fixed in 1.1.5.

[Bug 14639] ngen.exe from .NET 2.0 show exception E_INVALIDARG when trying to compile an assembly

http://bugs.winehq.org/show_bug.cgi?id=14639

Anastasius Focht  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
      Fixed by SHA1|                            |5c2fd1b192380fb741781db55b8
                   |                            |fb93c2a336810
           Hardware|Other                       |x86
                 OS|other                       |Linux