I have progressed a bit in attempts to load mixed code DLL into 64-bit process. To do this I built wine-mono pure assembly dlls (mscorlib and others) with -platform:x64 flag which provides x86_64 pure assembly dlls and thus allows to workaround DLL load issue.
Interesting. Do you have a patch to Mono that does this?
In this state ReallyFixupVTable is called from x86_64 thunk, where mono_marshal_get_vtfixup_ftnptr() starts executing 64-bit mixed code DLL's assembly initialization, but currently fails with my DLL on "invalid IL instruction" somewhere around System.IntPtr.ctor call (the same class works on i686), which is probably some mono issue. So I cannot come up with a fully working test case for now, though I did not give up yet.
I can try to build a 64-bit managed C++ dll in visual studio and call it from a C program.
If Mono still has trouble with that, I think there are ways to use vtable fixups in (otherwise) pure managed code to export methods to C.
+#elif __x86_64__ /* !defined(__i386__) */
Later in the patch it says defined(__x86_64__). I have no idea whether this distinction matters.
Comments say "defined", while there are #if's. It was originally like this when it was __i386__ only, I tried to minimize the changes. #if does work both for i386 and x86_64 (though #ifdef would work either). Should I fix comments for consistency?
I guess it's not important then. If you're touching those lines anyway, I guess it makes sense to fix it.
But more importantly, I don't think we should commit this until we can test it somehow.