Hi.
I think this part of d3dfmt_p8_init_palette has a bug, because it appears to be checking that render target is a render target. Incorrectly going into index_in_alpha codepath here causes this bug:
http://bugs.winehq.org/show_bug.cgi?id=10797
in Aliens vs Predator 1 game - it prefers P8 textures and sets color key on some textures for transparency and alpha channel is wrong on such textures in game.
dlls/wined3d/surface.c | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-)
Hi,
Unfortunately this patch is not correct and I haven't looked exactly what the real issue is.
In short most hardware (except if it has GL_EXT_paletted_texture) doesn't support 8bit textures and most of the time we do 8bit->32bit conversion in software. On hardware that supports fragment shaders we perform the conversion using a shader and this is a lot faster (it is 'free'). The palette is uploaded in d3dfmt_p8_init_palette.
So the p8 conversion is done in hardware but in some cases we need to read back the data from the card. The p8 -> 32bit conversion was done behind the back of the game, so the game expects 8bit data back. A palette contains upto 256 entries and each color can appear multiple times in the palette. So the inverse palette lookup doesn't have a 1:1 mapping and this is problematic.
The trick we do is to store the 8bit color index in the alpha component of the palette and use this to read back the correct index during the 32bit->8bit conversion. The problem is that most games need alpha and for that reason we only use this 'hack' when the primary render target is using 8bit. In this case you can't be using 3D and only old apps like StarCraft use this.
Your game is likely using some p8 textures and is likely suffering from the 32 -> 8bit problem. I haven't check AVP but if this is the real issue there it can't be fixed. It would break other games like GTA4 and others.
Regards, Roderick Colenbrander
Hi.
Yes, exactly, the AVP1 game is using p8 textures and color keys on some of them for transparency.
The thing is, the "index_in_alpha" hack DOES get enabled on its P8 3D textures, and that's the problem I'm trying to fix. I don't see the "primary render target is p8" check in the code, instead it looks like it checks that render_targets[0] is a render target.
In patch I proposed I check that the passed surface is a render target, and, since it's passed to d3dfmt_p8_init_palette then I guess it means there's a p8 render target and in such case the index_in_alpha hack is enabled. Although I don't check anything beyond just that it's a render target, I guess it could be some offscreen buffer. Is that the problem, essentially?
Regarding GTA - I could try looking for demo for tests, but can you clarify - is it really GTA4? GTA4 is upcoming game AFAIK, maybe you mean GTA3?
Roderick Colenbrander wrote:
Hi,
Unfortunately this patch is not correct and I haven't looked exactly what the real issue is.
In short most hardware (except if it has GL_EXT_paletted_texture) doesn't support 8bit textures and most of the time we do 8bit->32bit conversion in software. On hardware that supports fragment shaders we perform the conversion using a shader and this is a lot faster (it is 'free'). The palette is uploaded in d3dfmt_p8_init_palette.
So the p8 conversion is done in hardware but in some cases we need to read back the data from the card. The p8 -> 32bit conversion was done behind the back of the game, so the game expects 8bit data back. A palette contains upto 256 entries and each color can appear multiple times in the palette. So the inverse palette lookup doesn't have a 1:1 mapping and this is problematic.
The trick we do is to store the 8bit color index in the alpha component of the palette and use this to read back the correct index during the 32bit->8bit conversion. The problem is that most games need alpha and for that reason we only use this 'hack' when the primary render target is using 8bit. In this case you can't be using 3D and only old apps like StarCraft use this.
Your game is likely using some p8 textures and is likely suffering from the 32 -> 8bit problem. I haven't check AVP but if this is the real issue there it can't be fixed. It would break other games like GTA4 and others.
Regards, Roderick Colenbrander
Hi.
Yes, exactly, the AVP1 game is using p8 textures and color keys on some of them for transparency.
The thing is, the "index_in_alpha" hack DOES get enabled on its P8 3D textures, and that's the problem I'm trying to fix. I don't see the "primary render target is p8" check in the code, instead it looks like it checks that render_targets[0] is a render target.
Try to see if the attached patch fixes the problem. It seems that at some point I didn't check whether the target is P8 in this function while I check for it in other places. I have checked it using all my p8 testcases and it seems to work fine but I don't have AVP. If it works I'll submit it.
Roderick
Yes your patch fixes most problems (there are some remaining issues but they were not fixed by mine either).
Roderick Colenbrander wrote:
Hi.
Yes, exactly, the AVP1 game is using p8 textures and color keys on some of them for transparency.
The thing is, the "index_in_alpha" hack DOES get enabled on its P8 3D textures, and that's the problem I'm trying to fix. I don't see the "primary render target is p8" check in the code, instead it looks like it checks that render_targets[0] is a render target.
Try to see if the attached patch fixes the problem. It seems that at some point I didn't check whether the target is P8 in this function while I check for it in other places. I have checked it using all my p8 testcases and it seems to work fine but I don't have AVP. If it works I'll submit it.
Roderick
-
Alexander Dorofeyev -
Roderick Colenbrander