http://bugs.winehq.org/show_bug.cgi?id=10992
--- Comment #27 from Stefan Dösinger stefandoesinger@gmx.at 2008-03-04 16:06:36 --- This return value is D3DERR_INVALIDCALL. It's the general return value if something is wrong.
Does the game use ddraw.dll, d3d8.dll or d3d9.dll?
It might be possible as well that our d3d8.dll converts the d3d8 declaration incorrectly to the wined3d decl, or that there is some undocumented stuff when using them with the fixed function pipeline. In d3d8 there are no FVFs or Vertex Declarations like in d3d9. There are only vertex shaders, which in this case are 32 bit integers(HANDLEs). Shader handles smaller than the biggest possible FVF code are treated as an FVF code. Other shaders can be created using CreateVertexShaders. Shaders have a function and a declaration. If the function pointer is NULL, then the shader serves just as a declaration, similarly to a Direct3D9 vertex declaration.
My quick guess is that if there's a d3d8 shader without a function, then custom attribute types are ignored and d3d8 always uses the standard type for each usage type.(e.g. DCLUSAGE_COLOR -> D3DVSDT_D3DCOLOR, DCLUSAGE_POSITION -> FLOAT3, POSITIONT -> FLOAT4, etc)