In terms of core shader runner infrastructure, I don't think there's much preventing any of these? I'd prefer run_shader_tests_d3d12() taking care of which configurations to run internally instead of the main shader runner making that decision for us [...]
The implementation in !434 iterates over all shader models, but skips when the model is not required, if it is, runner's check_requirements() is used to specify if it supports that shader model. So we would be running tests using all shader models that are enumerated in the tests, for all runners that support it.
Right. The idea (and this was mostly my fault) is that in practice, the given constraints end up being equivalent to
``` for target in (sm1, sm4, sm6) for each backend run the tests ```
where a runner skips tests that don't make sense for it [which I think is necessary anyway], i.e. d3d9 skips anything other than sm1; d3d11 skips sm6 (and currently also sm4), etc. So this is a bit simpler than listing out every combination. It also means that we can get "run for every version" easily by just adjusting the outer loop.
That one is probably a little harder. I'm not sure "[require]" is the ideal place for this, but I imagine fundamentally we want something equivalent to this: [...] or some other variant. In any case, it doesn't seem ideal to me to impose explicitly listing individual shader models on every test if there are only a limited number of tests that need this.
Well, !434 adds a little verbosity to the tests, but I don't think it is so bad. Also, to avoid writing so many [require] directives, we make the defaults as if there was a:
[require] shader model 2.0 shader model 4.0 shader model 6.0at the beginning of the test files.
Yes, the idea is that there's an inherent default of (1, 4, 6) or something similar, so we wouldn't need to specify everywhere.