Wined3d/OpenGL question

[Jason Green]

I may be barking up the wrong tree here, but I'm trying to debug the
cause of a 502 error ( GL_INVALID_OPERATION ) from glDrawElements() in
drawprim.c (line 1251).  This is for Civilization 4 using both
hardware vertex & pixel shaders.  I've only noticed this error when it
is passed GL_TRIANGLES with a large number of vertices.

Here's a sample output with a little extra debugging info, and cleaned
up a little:

trace:d3d_draw:primitiveToGl TRIANGLES
trace:d3d_draw:drawPrimitiveDrawStrided begin number of indices6144,
number of vertices1156
trace:d3d_draw:drawPrimitiveDrawStrided end number of indices6144,
number of vertices1156
...
trace:d3d_draw:drawPrimitiveDrawStrided Using pixel shader
trace:d3d_draw:drawPrimitiveDrawStrided
glBindProgramARB(GL_FRAGMENT_PROGRAM_ARB, pixelShader->prgId); call ok
drawprim.c / 1846
trace:d3d_draw:drawPrimitiveDrawStrided
glEnable(GL_FRAGMENT_PROGRAM_ARB); call ok drawprim.c / 1850
trace:d3d_draw:drawStridedFast (0x7fd98890) : glElements(4, 6144, 0, ...)
fixme:d3d_draw:drawStridedFast >>>>>>>>>>>>>>>>> 502 from
glDrawRangeElements @ drawprim.c / 1251

Often, a series of these errors will result in an unhandled exception
in libglcore.so.1 (nvidia binary 8178 driver for a 6600GT, 2.6.16.1
kernel).

I read up on the glDrawElements() function and it refered to the
GetIntegerv() function for GL_MAX_ELEMENTS_VERTICES.   When I call
that function, it returns 4096, and the number of vertices that it's
trying to pass is larger than that.  I've tried doing a min() of the
max and the actual # of vertices, but the results are the same.

So, the question is:  Could this be one of the causes of the graphical
glitches I'm seeing?  I can toy around with these functions all day,
but I don't really understand them all that well.  Does that routine
need to be reworked to split up a D3DPT_TRIANGLELIST into multiple
calls if the calculated number of vertices/indices is too high for the
user's graphics card?  Or, am I way off base thinking this has any
bearing on my real issues?

Thanks,
Jason

[H. Verbeet]

I think the first thing you should do, is to verify that it's the call
to glDrawElements() that's generating the errors. glGetError just
check if an error flag is set, but if an error flag is set it doesn't
neccesarily mean the call right before glGetError generated that
error. The easiest way to do this would probably be to call glGetError
*before* the call to glDrawElements and verify that it return 0 /
GL_NO_ERROR.

[Jason Green]

On 4/4/06, H. Verbeet <hverbeet(a)gmail.com> wrote:
> I think the first thing you should do, is to verify that it's the call
> to glDrawElements() that's generating the errors. glGetError just
> check if an error flag is set, but if an error flag is set it doesn't
> neccesarily mean the call right before glGetError generated that
> error. The easiest way to do this would probably be to call glGetError
> *before* the call to glDrawElements and verify that it return 0 /
> GL_NO_ERROR.

Well, added that call just before the glDrawElements() call, and it
returns "call ok".  So, I tried Jame's suggestion that it was stuck
inside of a glBegin()/glEnd() call...  It didn't look like it in the
source, but to be sure, I put an extra glEnd() in the code under
certain circumstances [cleaned up and stripped of #ifdefs]:

    checkGLcall("before glDrawElements");
    /* This particular # of vertices w/ GL_TRIANGLES is on that is 
failing, so check for it */
    if (glPrimitiveType == 4 && numberOfVertices == 6144)
    {
        glEnd();
        checkGLcall("glEnd()");
    }
    if (idxData != NULL /* This crashes sometimes!*/) {
        TRACE("(%p) : glElements(%x, %d, %ld, ...)\n", This,
glPrimitiveType, numberOfVertices, minIndex);
        idxData = idxData == (void *)-1 ? NULL : idxData;
...
        glDrawElements(glPrimitiveType, numberOfVertices, idxSize == 2
? GL_UNSIGNED_SHORT : GL_UNSIGNED_INT,
                     (const char *)idxData+(idxSize * startIdx));
        checkGLcall("glDrawRangeElements");
    }
Here's the output:

trace:d3d_draw:drawStridedFast before glDrawElements call ok drawprim.c / 1226
fixme:d3d_draw:drawStridedFast >>>>>>>>>>>>>>>>> 502 from glEnd() @
drawprim.c / 1230
trace:d3d_draw:drawStridedFast (0x7fd8d1e8) : glElements(4, 6144, 0, ...)
fixme:d3d_draw:drawStridedFast >>>>>>>>>>>>>>>>> 502 from
glDrawRangeElements @ drawprim.c / 1248


I also did another check later to verify that idxData != NULL.  All I
can figure is that the data in idxData is somehow not matching up with
the data that OpenGL is looking for...  or the calculated # of
vertices or indices is somehow wrong?  I really don't know past this
point, but am willing to keep plugging away.  Any further ideas are
welcome.

Thanks

[H. Verbeet]

Just a guess, but could you check the program ids for the vertex/pixel
shaders in drawPrimitiveDrawStrided?

ie, prgId in http://source.winehq.org/git/?p=wine.git;a=blob;h=be8c038b273f2bffd4cb76ddc3...
and http://source.winehq.org/git/?p=wine.git;a=blob;h=be8c038b273f2bffd4cb76ddc3...

[Paul]

On Tue, 2006-04-04 at 13:37 -0400, Jason Green wrote:
> I may be barking up the wrong tree here, but I'm trying to debug the
> cause of a 502 error ( GL_INVALID_OPERATION ) from glDrawElements() in
> drawprim.c (line 1251).  This is for Civilization 4 using both
> hardware vertex & pixel shaders.  I've only noticed this error when it
> is passed GL_TRIANGLES with a large number of vertices.
>
> Here's a sample output with a little extra debugging info, and cleaned
> up a little:
>
> trace:d3d_draw:primitiveToGl TRIANGLES
> trace:d3d_draw:drawPrimitiveDrawStrided begin number of indices6144,
> number of vertices1156
> trace:d3d_draw:drawPrimitiveDrawStrided end number of indices6144,
> number of vertices1156
> ...
> trace:d3d_draw:drawPrimitiveDrawStrided Using pixel shader
> trace:d3d_draw:drawPrimitiveDrawStrided
> glBindProgramARB(GL_FRAGMENT_PROGRAM_ARB, pixelShader->prgId); call ok
> drawprim.c / 1846
> trace:d3d_draw:drawPrimitiveDrawStrided
> glEnable(GL_FRAGMENT_PROGRAM_ARB); call ok drawprim.c / 1850
> trace:d3d_draw:drawStridedFast (0x7fd98890) : glElements(4, 6144, 0, ...)
> fixme:d3d_draw:drawStridedFast >>>>>>>>>>>>>>>>> 502 from
> glDrawRangeElements @ drawprim.c / 1251
>
> Often, a series of these errors will result in an unhandled exception
> in libglcore.so.1 (nvidia binary 8178 driver for a 6600GT, 2.6.16.1
> kernel).

If anybody cares I get a segfault on a couple different nVidia drivers I
tried including 8178 using a 5700 Ultra. Been bitten by it in recent
versions of wine ... last one I tried that worked was 0.9.6 for halflife
2 and the windows verion of Quake 3.

Paul