On Fri, 2005-12-16 at 02:08 +0000, Aric Cyr wrote:
Aric Cyr <Aric.Cyr <at> gmail.com> writes:
I debugged it more last night and I found the offending call. If I comment out just that one (probably unneeded from what I can tell) function everything works great.
Replying to myself again...
Okay so I think this whole problem might turn out to be a compiler bug. I reverted all changes to drawprim.c I made to track down this bug, then I tried a something different. Since I was crashing in glDrawArrays() I was thinking that the vertex array or normal array was not properly set for some reason. Out of the 4 demos I test with, one of them is unaffected by my bug, and running with WINEDEBUG=d3d_draw I can see that the glDrawArrays path is not taken. So that narrowed the problem down to the glDrawArrays code path for me. In checking the glVertexPointer function call I changed the VTRACE message from
/* Note dwType == float3 or float4 == 2 or 3 */ VTRACE(("glVertexPointer(%ld, GL_FLOAT, %ld, %p)\n", sd->u.s.position.dwStride, sd->u.s.position.dwType + 1, sd->u.s.position.lpData));to
/* Note dwType == float3 or float4 == 2 or 3 */ VTRACE(("glVertexPointer(%u, %u, %ld, %p)\n", WINED3D_ATR_SIZE(position), WINED3D_ATR_GLTYPE(position), sd->u.s.position.dwStride, sd->u.s.position.lpData));After making this change everything started working... this is silly though since VTRACE is just a macro around TRACE(). So either there is stack corruption somewhere or the compiler is wonky. Since it is gcc-4.0.2 I'm guessing the compiler is buggy (Ubuntu breezy's standard gcc is 4.0.2 now). I'll try recompiling with gcc-3.3 later and see if the original code is working.
Still interested in any comments though :)
Hi Aric, If you see stack corruption like this, you might want to try compiling with optimization turned off. put the -O0 (a capital letter O followed by a zero) flag in your CFLAGS when you run configure. I had a similar situation where gcc was using fuzzy math when working with structures, and turning off optimization helped.
James