Stencil mask is supposed to be used with reference value on comparison according to MSDN [1] (which of course does not guarantee that it is the case on Windows). I've modified your test a bit to show that it is really the case, the modified test succeeds for me on Windows 7, Nvidia or Intet GPU. HP (see attached, to be applied after your patches). But it also succeeds in Wine with your modifications to wined3d (after removing AND with the mask). Which probably means that: - Opengl applies the mask to ref value as it is supposed to ([2]); - ref value is probably not clamped to stencil buffer range for comparison while it looks so in native d3d9; - Stencil mask is ignored when storing ref value to stencil buffer both in opengl and d3d9.
I also tested that Test Drive Unlimited glitch (which was fixed by a9e199e5ad76c0540381f25011d6051ace3e7c0a) does not reappear after your patches.
1. https://docs.microsoft.com/en-us/windows/win32/direct3d9/d3drenderstatetype, description for D3DRS_STENCILMASK. 2. https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glStencilFuncSepar..., description for 'mask' parameter.
On 8/21/19 12:46, Zhiyi Zhang wrote:
This proves that stencil reference got truncated instead of ANDed with stencil mask.
Signed-off-by: Zhiyi Zhang zzhang@codeweavers.com