Instead of introducing a flag that must be checked throughout the runner, I'd rather introduce another runner ops that just compiles and does nothing else.
Okay, but we would need a flag in the runner to discriminate whether it is a compilation-only runner, otherwise we must either define all the [test] related ops as empty functions or allowing them to be NULL, which means adding a lot of checks.
Also, it doesn't make sense to run pure compilation tests on crosstest and Windows builds, given that all the shader models are natively supported there anyway.
I am not sure I understand this, the Windows "non-cross" builds use vkd3d-shader, instead of the native compiler right?
Also, I think that it makes some sense to run compilation tests even with the native compiler, because we can make sure that we didn't made a mistake when writing the tests, or that fail/notimpl tests indeed fail.