Hi,
This is a brain core dump before going to bed for the other d3d developers. I did some testing concerning the vertexdeclaration, vertex shader, stream sources and pixel shader states. Currently the first 3 are hard linked, and the pixel shader is also tied to the vertex shader. I was interested which states are invalid how often, and where separating the states makes sense.
Basically about 40% of all draws only the pixel shader is invalid. In another 30%-50% the pixel shader and stream source are invalid, and in about 10% to 20% of all draws all 4 states are reapplied anyways. There are relatively few cases where the vertex declaration and vertex shader are changed alone.
So what does this mean? We'll have to spit away the pixel shader and stream sources states. The good news is that the stream sources are easy to separate - when the streamsources are invalid only reapply them, if the vdecl or vshader are dirty reapply shader+decl+streamsrc. The bad news is that applying the stream sources is the most expensive part.
The pixel shader is already somewhat separated. Changing the pixel shader will only reapply the vertex shader, but not stream sources or other vdecl-related states. Separating vertex and pixel shaders will be tricky and not fully possible on some drivers(needs GL_ARB_separate_shader_objects or ARB shaders) and all shaders(Shader Model 3.0 varying passing)
In addition to this we'll have to make parsing of the vertex declaration cheaper. I'll think of something... I'll also write some quick and dirty patches to see how much performance we really get out of these changes.
Stefan
PS: I attached my debug patch and the output for a few games.