On Thu Jan 19 10:53:42 2023 +0000, Zebediah Figura wrote:
I'm not sure I completely follow the logic. I guess the interesting
thing about the d3d9 and d3d11 tests is that it would be much easier to end up with the Windows D3D + vkd3d-shader configuration, while for the d3d12 tests you'd typically either have Windows D3D + d3dcompiler or vkd3d + vkd3d-shader. (Note though that Wine and the possibility of using native DLLs does complicate that picture a little.) That means we may need to do a better job of distinguishing between Windows D3D and wined3d/vkd3d, separately from distinguishing between d3dcompiler and vkd3d-shader, instead of lumping these together, but that might not be a bad thing anyway? I was thinking of a division between vkd3d+vkd3d-shader and native (d3d*+d3dcompiler), where "crosstest" means native and normal test means vkd3d. I hadn't really considered the idea of mixing and matching the two. It seems interesting enough to try, but the way v4 of this series does it seems inconsistent—if we're going to build the d3d11 runner to use vkd3d's HLSL compiler with native vkd3d, I'd expect the d3d12 runner from the same executalbe to be built the same way. Granted, there's a printed warning that's *not* what's happening... Ultimately we are probably going to have to abandon "crosstests" for the shader runner; they're not going to be expressive enough.
Yeah, d3d12 is different because for d3d9 and d3d11 we can't do other then using the native thing, while for d3d12 in principle we can use either the native library or our own implementation. For testing I usually feel rather maximalist, so I would consider it ideal to have a way to run all the possible combinations: * compile the shaders with vkd3d-shader and execute them with vkd3d (via its D3D12 interface): this is already happening on non-cross builds (though it's true that with my patch the shader runner would falsely pretend that they're being run with d3d12.dll, so I'll fix the message); * compile the shaders with vkd3d-shader and execute them with d3d12.dll: this is neither happening currently nor with my patch; * compile the shaders with vkd3d-shader and execute them with d3d11.dll: this is not currently happening, but would with my patch on non-cross builds; * compile the shaders with vkd3d-shader and execute them with d3d9.dll: this is not currently happening, but would with my patch on non-cross builds; * compile the shaders with vkd3d-shader and execute them with Vulkan (via the custom shader runner): this is neither happening currently nor with my patch; * compile the shaders with d3dcompiler_47.dll and execute them with vkd3d: this is neither happening currently nor with my patch; * compile the shaders with d3dcompiler_47.dll and execute them with d3d12.dll: this is already happening on cross builds; * compile the shaders with d3dcompiler_47.dll and execute them with d3d11.dll: this is already happening on cross builds; * compile the shaders with d3dcompiler_47.dll and execute them with d3d9.dll: this is already happening on cross builds; * compile the shaders with d3dcompiler_47.dll and execute them with Vulkan (this would involve another recompilation step DXBC->SPIR-V with vkd3d-shader): this is neither happening currently nor with my patch.
All of this of course only makes sense for Win32 (cross) builds. On Unix we can only compile with vkd3d-shader and execute with vkd3d or Vulkan (and we already do both). In principle one could also add other rows/columns to the matrix (i.e., use d3dcompiler_N.dll or d3d10.dll), but maybe that's not terribly meaningful. Also, for tests that are not the shader runner only four lines of those make sense, if I am not mistaken (specifically, compiling with either vkd3d-shader or d3dcompiler_47.dll and executing with either vkd3d or d3d12.dll).
As I said, I feel rather maximalist, so in principle I'd implement all the lines above (though I acknowledge that some lines make more sense than others). Possibly with some options to disable some of them at runtime, for testing speed reasons. Also I'd rig things so that the non-cross builds is able to run all the combinations in a single executable, for convenience reasons.
However, I don't think that this is in scope for this specific MR, so I would ask for it to be accepted as it is (modulo perhaps warnings and code style), and other combinations (if considered desirable) can be added later.