I would like something a bit more coherent than "fail_tpf", "skip_dxil", and "todo_6" though. How about something along the lines of "fail(shader model < 6.0)" and "todo(shader model >= 6.0)"? "skip_dxil" sounds like it should simply be a "shader model < 6.0" line in the "[require]" section.
I agree something more coherent would be nice, but that seems excessively verbose to me. I don't think we're going to need any more complex categories than "sm1", "sm4", and "sm6".
Wrt fail_tpf... what may make sense is, every time we have a require directive for a shader version, we try to compile it with every version anyway, and validate that HLSL compilation fails for versions outside of the range. Then that would become
[require] shader model >= 6.0
Although you do mention that some tests ICE the native compiler, so I'm not sure how to deal with that...
Somewhat relatedly, for some of those tests I don't even know how the relevant shader would be interpreted, so we may want an actual [test] directive following them.
"skip_dxil" sounds like it should simply be a "shader model < 6.0" line in the "[require]" section.
Ironically I started with this and thought it may to too restrictive; I'll need to reorder the tests which need this so they come last in their source file, but it's rare enough not to be an issue.
No, we should probably just change [require] so that it doesn't skip the whole file.
That or move the relevant tests to separate files. Embedding lots of tests in the same file is... the problem with it is that, somewhat unlike C, it's not the easiest thing to read; I blame the lack of syntax highlighting support.