http://bugs.winehq.org/show_bug.cgi?id=19400
--- Comment #13 from Nate Eldredge nate@thatsmathematics.com 2009-11-09 23:28:33 --- Hmm. Well, have a look at the following article: http://support.microsoft.com/kb/112641
It appears that ExtDeviceMode doesn't have a prototype. Instead, you are supposed to get its address from GetProcAddress (FWIW, that's what this application does) and cast it to type LPFNDEVMODE. I don't know how LPFNDEVMODE is defined, and I didn't find documentation for it, but it could be that it is a pointer to a stdcall function. Would it be kosher to examine the Windows header files to find out? Or alternatively, to compile and execute some code on a Windows machine to see what it does?
I'm inclined to suspect that stdcall is correct, because the application runs correctly under Windows (at least I assume it does, I don't have a real Windows box to test, but it's a well-regarded commercial app so I assume it isn't totally broken ;-). If it tried to call this function using the wrong calling conventions it would surely crash or otherwise fail (e.g. the stack sentinel I mentioned). So either the documentation is wrong and it is in fact stdcall, or the application has got it wrong but for some reason under Windows the erroneous code never runs (seems unlikely).
Similar issue with respect to the first two arguments. The documentation disagrees with what Wine does, but it looks like the application is expecting Wine's behavior (though I don't know enough to be sure). Again, could we check what real Windows does, in case it is misdocumented?
Thanks for looking at this. I'm new to wine hacking and it's good to have someone else thinking about the issue.