Direct3D stores vertex color data in BGRA order, OpenGL stores it in RGBA order. This causes red and blue to be swapped in d3d applications when run under wine. This patch fixes this problem for vertex shaders by enabling the flag that causes red and blue to be swapped in vshader_program_add_input_param_swizzle, when appropriate.
Changelog:
- Enable color fixups for vertex shaders
In my local tree I have some simple d3d8 shaders working. (See for instance http://roderick.student.utwente.nl/dolphin1.png and dolphin2.png on windows; there's some color issue but apart from that it works). This patch causes in combination with the other vshader patch you sent some drawing issues. Check http://roderick.student.utwente.nl/dolphin3.png
Regards, Roderick
On 09/03/06, Roderick Colenbrander thunderbird2k@gmx.net wrote:
In my local tree I have some simple d3d8 shaders working. (See for instance http://roderick.student.utwente.nl/dolphin1.png and dolphin2.png on windows; there's some color issue but apart from that it works). This patch causes in combination with the other vshader patch you sent some drawing issues. Check http://roderick.student.utwente.nl/dolphin3.png
Any idea why that could be happening?
In particular, is it the fixup for DIFFUSE, SPECULAR or either that causes the problem? What are the values for reg, arrayUsageMap[WINED3DSHADERDECLUSAGE_DIFFUSE] and arrayUsageMap[WINED3DSHADERDECLUSAGE_SPECULAR]?
It looks like texture coordinates are wrong, but those shouldn't be affected by this patch, do you initialise arrayUsageMap correctly?
In particular, is it the fixup for DIFFUSE, SPECULAR or either that causes the problem? What are the values for reg, arrayUsageMap[WINED3DSHADERDECLUSAGE_DIFFUSE] and arrayUsageMap[WINED3DSHADERDECLUSAGE_SPECULAR]?
It looks like texture coordinates are wrong, but those shouldn't be affected by this patch, do you initialise arrayUsageMap correctly?
I have walked through the code. This small demo (dolphinvs from dx8.1 sdk) uses the following vertex declaration for the seafloor: D3DVSD_STREAM( 0 ), D3DVSD_REG( 0, D3DVSDT_FLOAT3 ), // Position of first mesh D3DVSD_REG( 3, D3DVSDT_FLOAT3 ), // Normal D3DVSD_REG( 6, D3DVSDT_FLOAT2 ), // Tex coords D3DVSD_END()
It stores texture coordinates in register 6 (D3DVSD_SPECULAR). Registers should be general purpose I think and this should work fine. Your patch assumes that 'D3DVSD_SPECULAR' is allways used for colors which should be swizzled. Because we are storing texture coordinates in them they shouldn't be swizzled but we don't know what is stored in them. I'm not sure how to fix this bug but I think the code should be moved to a later stage. Swizzling won't be done in the shader then :(
Roderick
On 10/03/06, Roderick Colenbrander thunderbird2k@gmx.net wrote:
I have walked through the code. This small demo (dolphinvs from dx8.1 sdk) uses the following vertex declaration for the seafloor: D3DVSD_STREAM( 0 ), D3DVSD_REG( 0, D3DVSDT_FLOAT3 ), // Position of first mesh D3DVSD_REG( 3, D3DVSDT_FLOAT3 ), // Normal D3DVSD_REG( 6, D3DVSDT_FLOAT2 ), // Tex coords D3DVSD_END()
It stores texture coordinates in register 6 (D3DVSD_SPECULAR). Registers should be general purpose I think and this should work fine. Your patch assumes that 'D3DVSD_SPECULAR' is allways used for colors which should be swizzled.
Well, no. It assumes that arrayUsageMap[WINED3DSHADERDECLUSAGE_SPECULAR] contains the register number for the register that is used to store specular colors, or -1 if there are no specular colors. loadNumberedArrays() in drawprim.c uses arrayUsageMap in the same way.
Because we are storing texture coordinates in them they shouldn't be swizzled but we don't know what is stored in them. I'm not sure how to fix this bug but I think the code should be moved to a later stage.
I don't think that will work. We really need to know if a certain block of data is going to be used for colors or not, or we can't fix the data, in the shader or otherwise. Also, we have to fix the data before it gets passed to the shader or on input, we can't do it on output because other components (with the correct color order) might have been added in the shader.
Either way, arrayUsageMap doesn't get initialised correctly. If we don't know what's in a register, we shouldn't pass it through arrayUsageMap, but use some other way.
I won't be around next week, but after that I'll probably try to have a look at d3d8 shaders.
Either way, arrayUsageMap doesn't get initialised correctly. If we don't know what's in a register, we shouldn't pass it through arrayUsageMap, but use some other way.
I won't be around next week, but after that I'll probably try to have a look at d3d8 shaders.
The declaration is converted to a d3d9 shader in parsedeclaration8. The function parsetoken8 handles the conversion of a token. In this case the following code is entered:
case D3DVSD_TOKEN_STREAMDATA: if (token & 0x10000000) { TRACE(" 0x%08lx SKIP(%lu)\n", token, ((token & D3DVSD_SKIPCOUNTMASK)
D3DVSD_SKIPCOUNTSHIFT));
} else { DWORD type = ((token & D3DVSD_DATATYPEMASK) >> D3DVSD_DATATYPESHIFT); DWORD reg = ((token & D3DVSD_VERTEXREGMASK) >> D3DVSD_VERTEXREGSHIFT); ERR("reg: %d\n", reg); TRACE(" 0x%08lx REG(%s, %s)\n", token, VertexDecl8_Registers[reg], VertexDecl8_DataTypes[type]); }
Because reg is set to 6 in the declaration, this translates to specular because VertexDecl8_Registers[6] corresponds to specular. Someone in #winehackers thought that perhaps your code should only be called for FVF shaders.
Roderick
On 10/03/06, Roderick Colenbrander thunderbird2k@gmx.net wrote:
Because reg is set to 6 in the declaration, this translates to specular because VertexDecl8_Registers[6] corresponds to specular.
Yes, but that shouldn't make it into the arrayUsageMap. ArrayUsageMap defines usage, but in this case we don't know how the data will be used.
-
H. Verbeet -
Roderick Colenbrander