James Liggett <jrliggett <at> cox.net> writes:
Hi Aric, If you see stack corruption like this, you might want to try compiling with optimization turned off. put the -O0 (a capital letter O followed by a zero) flag in your CFLAGS when you run configure. I had a similar situation where gcc was using fuzzy math when working with structures, and turning off optimization helped.
Thanks for the suggestion, but I already have wine compiled with only "-g". Last time I checked gcc doesn't enable optimizations unless a -O option is specified explicitly (don't know if this has changed though).
Anyways, I think I finally figured this all out. As I thought it turned out to be stack corruption, due to differing calling conventions. It just required fixing up my function pointers to be WINAPI (which I believe becomes stdcall) for all the wgl functions.
Now everything works fine, except this brings up another issue with wglGetProcAddress. The problem is that all gl extensions function pointers are declared WINAPI, and indeed this is what type of functoin wglGetProcAddress is expected to return. However for extensions that are not registered in opengl_ext.c wglGetProcAddress falls back to glXGetProcAddressARB which would return a non-WINAPI function pointer. After the first call to such a function we would have corrupted the stack. There doesn't seem to be a nice way to fix this that I can think of. wglGetProcAddress should never return a non-WINAPI function though, as that is just asking for trouble. Better to return NULL instead of falling back to glXGetProcAddressARB.
As an aside to the wglGetProcAddress issue, all GL extenions called by wined3d are now passed through the thunks. It is only one extra function call so the impact should be minimal (too bad we can't inline these!), but I thought I should mention it anyways.
After cleaning up everything I've worked on I think I'll start submitting patches. First some pre-requisite patches for dlls/opengl32 and include/ and then the changes for dlls/wined3d. All together it is quite a large change, but I should be able to break it up into manageable pieces (hopefully!).
Regards, Aric