I disagree, and perhaps unsurprisingly in that regard, I don't think framing things in terms of "backend" and "frontend" is entirely fortunate. The way I look at it is that the different shader runners execute .shader_test files, in a number of different configurations, and we have some common parser (among others) infrastructure to help with that.
I don't understand this. My entire idea behind the shader runner was that it'd abstract away the difference between d3d APIs. But that just means that it abstracts away the implementation details. I don't see what's gained by giving individual runners the freedom to control *every* configuration decision, either conceptually or practically. Something like "are we using spirv?" certainly only makes sense in the context of one runner, but "shader model" is generic to the HLSL.
"Orthogonality is a good thing" is certainly a decent chunk of it, and conceptually I just don't think this is something that belongs in the parser, but ultimately I think it comes down to the fact that I think it works fine the way it currently works, and I don't see a compelling reason to change it. Perhaps that just means I'm oblivious to the problem, and perhaps that's something that was discussed in detail in person, but I did ask a couple of times, and didn't get a compelling answer.
Incidentally, I'm not sure HLSL should be quite as central to the shader runner either; I could certainly imagine that being able to specify shaders as d3d-asm or one of the supported bytcode formats (including compiled effect files) would have its uses.
It should be entirely up to individual runners to decide which configurations to run, and ideally this would be entirely opaque to the parser.
How do we specify whether tests should be run with e.g. every shader model or just a few?
I suspect your answer is "we don't; we run every test with every shader model". I haven't tried this but I worry this is going to become too many tests to run. Not least because I know Mesa has this problem.
There's other things that fall into this same category of "don't inhere to the backend, matter for some tests but not all, and would probably result in too many configurations if we tested all of them all the time". Two examples are compilation flags, and d3dcompiler/d3dx9 version.
We invent some syntax for it and then implement it. E.g.: ``` [test] foreach (sm1,sm3,sm5) draw quad probe ... ... done ``` The specific syntax isn't terribly important at this point, but the parser would essentially just unroll the loop in that case.
The broader, more important point though is that most of what's in this MR just shouldn't be needed for d3dbc support.