[PATCH 0/3] MR514: tests: Use the vulkan runner to run SM1 compilation tests.

[Francisco Casas (＠fcasas)]

Recap:

My previous proposals to extend the shader_runner to SM1 were focusing on separating compilation tests ([shader] directives) from execution tests ([test] directives), and let the generic shader_runner.c:compile_shader() in charge of the former.

However, Henri was more inclined to aim for making the parser agnostic to the language of the tests so we can potentially extend the shader_runner tests to other languages such as d3d-asm. This means, removing the burden of compilation from the parser altogether, and moving it to the runners, probably through a runner->compile function pointer (even if most of them end up doing the same thing, compiling with vkd3d-shader or the native compiler, depending on availability).

Agreeing with going in this general direction, this MR contains patches to do SM1 compilation calling run_shader_tests() for SM1 profiles from the Vulkan runner (skipping execution for now), and some improvements to the qualifiers syntax.

Despite this, there are a parallel discussion on:
- Whether to name the shader models individually in the [require] directives or expressing the range of shader models where the test should pass. In the latter case, whether to run for all shader models, only the lowest one in the SM1, SM4, and SM6 groups, or to allow the runner to select a shader model within the range (I feel strongly against this now, I think the runner should just retrieve a boolean in runner->check_requirements). 
- Where to do the iteration across different shader models, if in run_shader_tests() or expect each runner to call run_shader_tests() several times as we do now.
But there is no need to settle on something yet.

---

Now, what may be the most noisy part of this MR is that even though we are calling run_shader_tests() through the Vulkan runner, this will result in calling shader_runner.c:compile_shader() instead of shader_runner_vulkan.c:compile_shader(), but if we follow the "making the parser agnostic" plan, we eventually should get rid of shader_runner.c:compile_shader() and introduce the runner->compile pointer instead.

[PATCH 1/3] tests/shader-runner: Support reading multiple model range args for qualifiers.

[Francisco Casas]

From: Francisco Casas

We can now pass (sm<4) and (sm>=4) to "fail" and "todo" qualifiers, and
we also we can pass a specific range using two arguments separated
with commas, if we need them, e.g. todo(sm>=4,sm<6).

Adding additional model ranges for the tests, if we need them, should be
easier now, since they only have to be added to the "valid_args" array.
---
 tests/shader_runner.c | 160 +++++++++++++++++++++++++++++++++---------
 1 file changed, 125 insertions(+), 35 deletions(-)

diff --git a/tests/shader_runner.c b/tests/shader_runner.c
index 0e6104ce3..12af65252 100644
--- a/tests/shader_runner.c
+++ b/tests/shader_runner.c
@@ -94,42 +94,133 @@ enum parse_state
     STATE_TEST,
 };

-static bool match_string(const char *line, const char *token, const char **const rest)
+static void read_string_args(const char *line, const char **const rest,
+        enum shader_model *sm_min, enum shader_model *sm_max)
+{
+    unsigned int i;
+
+    static const struct
+    {
+        const char *text;
+        enum shader_model sm_min, sm_max;
+    }
+    valid_args[] =
+    {
+        {"sm>=4", SHADER_MODEL_4_0, SHADER_MODEL_6_0},
+        {"sm>=6", SHADER_MODEL_6_0, SHADER_MODEL_6_0},
+        {"sm<4",  SHADER_MODEL_2_0, SHADER_MODEL_4_0 - 1},
+        {"sm<6",  SHADER_MODEL_2_0, SHADER_MODEL_6_0 - 1},
+    };
+
+    assert(*line == '(');
+    ++line;
+
+    while (*line != ')')
+    {
+
+        for (i = 0; i < ARRAY_SIZE(valid_args); ++i)
+        {
+            const char *option_text = valid_args[i].text;
+
+            if (!strncmp(line, option_text, strlen(option_text)))
+            {
+                line += strlen(option_text);
+                *sm_min = max(*sm_min, valid_args[i].sm_min);
+                *sm_max = min(*sm_max, valid_args[i].sm_max);
+
+                break;
+            }
+        }
+
+        if (*line == ',')
+        {
+            ++line;
+        }
+        else if (*line != ')')
+        {
+            fatal_error("Invalid qualifier argument '%s'.\n", line);
+            return;
+        }
+    }
+
+    assert(*line == ')');
+    if (rest)
+        *rest = line + 1;
+}
+
+static bool match_string_with_args(const char *line, const char *token, const char **const rest,
+    enum shader_model *sm_min, enum shader_model *sm_max)
 {
     size_t len = strlen(token);

+    assert(!!sm_max == !!sm_min);
+
     while (isspace(*line))
         ++line;

-    if (strncmp(line, token, len) || !isspace(line[len]))
+    if (strncmp(line, token, len) || !(isspace(line[len]) || line[len] == '('))
         return false;
+    line += len;
+
+    if (sm_min)
+    {
+        *sm_min = SHADER_MODEL_2_0;
+        *sm_max = SHADER_MODEL_6_0;
+
+        if (*line == '(')
+            read_string_args(line, &line, sm_min, sm_max);
+    }
+
     if (rest)
     {
-        *rest = line + len;
+        *rest = line;
         while (isspace(**rest))
             ++*rest;
     }
     return true;
 }

-static bool match_directive_substring(const char *line, const char *token, const char **const rest)
+static bool match_string(const char *line, const char *token, const char **const rest)
+{
+    return match_string_with_args(line, token, rest, NULL, NULL);
+}
+
+static bool match_directive_substring_with_args(const char *line, const char *token,
+        const char **const rest, enum shader_model *sm_min, enum shader_model *sm_max)
 {
     size_t len = strlen(token);

while (isspace(*line) || *line == ']')
         ++line;

-    if (strncmp(line, token, len) || !(isspace(line[len]) || line[len] == ']'))
+    if (strncmp(line, token, len) || !(isspace(line[len]) || line[len] == ']' || line[len] == '('))
         return false;
+    line += len;
+
+    if (sm_min)
+    {
+        assert(sm_max);
+        *sm_min = SHADER_MODEL_2_0;
+        *sm_max = SHADER_MODEL_6_0;
+
+        if (*line == '(')
+            read_string_args(line, &line, sm_min, sm_max);
+    }
+
     if (rest)
     {
-        *rest = line + len;
+        *rest = line;
         while (isspace(**rest))
             ++*rest;
     }
     return true;
 }

+static bool match_directive_substring(const char *line, const char *token, const char **const rest)
+{
+    return match_directive_substring_with_args(line, token, rest, NULL, NULL);
+}
+
 static void parse_require_directive(struct shader_runner *runner, const char *line)
 {
     bool less_than = false;
@@ -535,17 +626,17 @@ static void read_uint64_t2(const char **line, struct u64vec2 *v)

static void parse_test_directive(struct shader_runner *runner, const char *line)
 {
+    enum shader_model sm_range_min, sm_range_max;
     char *rest;
     int ret;

runner->is_todo = false;

-    if (match_string(line, "todo", &line))
-        runner->is_todo = true;
-    else if (match_string(line, "todo(sm<6)", &line))
-        runner->is_todo = runner->minimum_shader_model < SHADER_MODEL_6_0;
-    else if (match_string(line, "todo(sm>=6)", &line))
-        runner->is_todo = runner->minimum_shader_model >= SHADER_MODEL_6_0;
+    if (match_string_with_args(line, "todo", &line, &sm_range_min, &sm_range_max))
+    {
+        if (sm_range_min <= runner->minimum_shader_model && runner->minimum_shader_model <= sm_range_max)
+            runner->is_todo = true;
+    }

if (match_string(line, "dispatch", &line))
     {
@@ -1059,42 +1150,41 @@ static void compile_shader(struct shader_runner *runner, IDxcCompiler3 *dxc_comp
     }
 }

+static enum parse_state get_parse_state_todo(enum parse_state state)
+{
+    if (state == STATE_SHADER_COMPUTE)
+        return STATE_SHADER_COMPUTE_TODO;
+    else if (state == STATE_SHADER_PIXEL)
+        return STATE_SHADER_PIXEL_TODO;
+    else if (state == STATE_SHADER_VERTEX)
+        return STATE_SHADER_VERTEX_TODO;
+    else
+        return STATE_SHADER_EFFECT_TODO;
+}
+
 static enum parse_state read_shader_directive(struct shader_runner *runner, enum parse_state state, const char *line,
         const char *src, HRESULT *expect_hr)
 {
+    enum shader_model sm_range_min, sm_range_max;
+
     while (*src && *src != ']')
     {
-        /* 'todo' is not meaningful when dxcompiler is in use, so it has no '(sm<6) qualifier. */
-        if (match_directive_substring(src, "todo", &src))
+        if (match_directive_substring_with_args(src, "todo", &src, &sm_range_min, &sm_range_max))
         {
+            /* 'todo' is not meaningful when dxcompiler is in use. */
             if (runner->minimum_shader_model >= SHADER_MODEL_6_0)
                 continue;
-            if (state == STATE_SHADER_COMPUTE)
-                state = STATE_SHADER_COMPUTE_TODO;
-            else if (state == STATE_SHADER_PIXEL)
-                state = STATE_SHADER_PIXEL_TODO;
-            else if (state == STATE_SHADER_VERTEX)
-                state = STATE_SHADER_VERTEX_TODO;
-            else
-                state = STATE_SHADER_EFFECT_TODO;
-        }
-        else if (match_directive_substring(src, "fail", &src))
-        {
-            *expect_hr = E_FAIL;
+            if (sm_range_min <= runner->minimum_shader_model && runner->minimum_shader_model <= sm_range_max)
+                state = get_parse_state_todo(state);
         }
-        else if (match_directive_substring(src, "fail(sm<6)", &src))
+        else if (match_directive_substring_with_args(src, "fail", &src, &sm_range_min, &sm_range_max))
         {
-            if (runner->minimum_shader_model < SHADER_MODEL_6_0)
-                *expect_hr = E_FAIL;
-        }
-        else if (match_directive_substring(src, "fail(sm>=6)", &src))
-        {
-            if (runner->minimum_shader_model >= SHADER_MODEL_6_0)
+            if (sm_range_min <= runner->minimum_shader_model && runner->minimum_shader_model <= sm_range_max)
                 *expect_hr = E_FAIL;
         }
-        else if (match_directive_substring(src, "notimpl(sm<6)", &src))
+        else if (match_directive_substring_with_args(src, "notimpl", &src, &sm_range_min, &sm_range_max))
         {
-            if (runner->minimum_shader_model < SHADER_MODEL_6_0)
+            if (sm_range_min <= runner->minimum_shader_model && runner->minimum_shader_model <= sm_range_max)
                 *expect_hr = E_NOTIMPL;
         }
         else

[PATCH 2/3] tests/shader-runner: Allow specifying multiple todo() qualifiers on test directives.

[Francisco Casas]

From: Francisco Casas

So if we have tests that are todo on both sm<4 and sm>=6 we can write,
e.g.:

todo(sm<4) todo(sm>=6) draw draw

OTOH if we were to have tests that are todo only in sm>=4 and sm<6 we
can write:

todo(sm>=4,sm<6) draw quad

Effectively, multiple arguments in a single todo() are like an AND and
multiple todo()-s behave like an OR.
---
 tests/shader_runner.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/shader_runner.c b/tests/shader_runner.c
index 12af65252..ec6aac427 100644
--- a/tests/shader_runner.c
+++ b/tests/shader_runner.c
@@ -632,7 +632,7 @@ static void parse_test_directive(struct shader_runner *runner, const char *line)

runner->is_todo = false;

-    if (match_string_with_args(line, "todo", &line, &sm_range_min, &sm_range_max))
+    while (match_string_with_args(line, "todo", &line, &sm_range_min, &sm_range_max))
     {
         if (sm_range_min <= runner->minimum_shader_model && runner->minimum_shader_model <= sm_range_max)
             runner->is_todo = true;

[PATCH 3/3] tests: Use the vulkan runner to run SM1 compilation tests.

[Francisco Casas]

From: Francisco Casas

At the current moment this is a little odd because for SM1 [test]
directives are skipped, and the [shader] directives are not executed by
the shader_runner_vulkan.c:compile_shader() but by the general
shader_runner.c:compile_shader(). So in principle it is a little weird
that we go through the vulkan runner.

But fret not, because in the future we plan to make the parser agnostic
to the language of the tests, so we will get rid of the general
shader_runner.c:compile_shader() function and instead call a
runner->compile_shader() function, defined for each runner. Granted,
most of these may call a generic implementation that uses native
compiler in Windows, and vkd3d-shader on Linux, but it would be more
conceptually correct.
---
 tests/hlsl/any.shader_test                    |  4 +-
 .../hlsl/arithmetic-float-uniform.shader_test |  8 ++--
 tests/hlsl/arithmetic-int-uniform.shader_test | 16 ++++----
 tests/hlsl/arithmetic-int.shader_test         |  2 +
 tests/hlsl/ceil.shader_test                   |  2 +-
 tests/hlsl/combined-samplers.shader_test      |  4 +-
 tests/hlsl/conditional.shader_test            |  8 ++--
 tests/hlsl/discard.shader_test                |  2 +-
 tests/hlsl/distance.shader_test               |  2 +-
 tests/hlsl/expr-indexing.shader_test          |  6 +--
 tests/hlsl/float-comparison.shader_test       |  2 +-
 tests/hlsl/floor.shader_test                  |  2 +-
 tests/hlsl/fmod.shader_test                   |  4 +-
 tests/hlsl/for.shader_test                    |  4 +-
 tests/hlsl/function-return.shader_test        | 14 +++----
 tests/hlsl/half.shader_test                   |  2 +-
 tests/hlsl/hard-copy-prop.shader_test         |  7 ++--
 tests/hlsl/lit.shader_test                    |  4 +-
 tests/hlsl/loop.shader_test                   | 14 +++----
 tests/hlsl/matrix-indexing.shader_test        |  4 +-
 tests/hlsl/non-const-indexing.shader_test     | 12 +++---
 tests/hlsl/return.shader_test                 | 23 ++++--------
 tests/hlsl/round.shader_test                  |  4 +-
 tests/hlsl/sample-bias.shader_test            |  2 +-
 tests/hlsl/sample-level.shader_test           |  2 +-
 tests/hlsl/sampler.shader_test                |  4 +-
 tests/hlsl/saturate.shader_test               |  2 +-
 tests/hlsl/sign.shader_test                   | 12 +++---
 tests/hlsl/smoothstep.shader_test             |  2 +-
 tests/hlsl/static-initializer.shader_test     |  5 ++-
 tests/hlsl/step.shader_test                   |  2 +-
 tests/hlsl/trigonometry.shader_test           |  8 ++--
 tests/hlsl/trunc.shader_test                  |  4 +-
 .../hlsl/vector-indexing-uniform.shader_test  |  2 +-
 tests/shader_runner.c                         | 37 +++++++++++--------
 tests/shader_runner.h                         |  6 ++-
 tests/shader_runner_d3d11.c                   |  2 +-
 tests/shader_runner_d3d12.c                   |  2 +-
 tests/shader_runner_d3d9.c                    |  2 +-
 tests/shader_runner_gl.c                      |  2 +-
 tests/shader_runner_vulkan.c                  | 16 +++++---
 41 files changed, 134 insertions(+), 128 deletions(-)

diff --git a/tests/hlsl/any.shader_test b/tests/hlsl/any.shader_test
index f2298d3a3..27a16a50d 100644
--- a/tests/hlsl/any.shader_test
+++ b/tests/hlsl/any.shader_test
@@ -1,4 +1,4 @@
-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 f;

float4 main() : sv_target
@@ -29,7 +29,7 @@ uniform 0 float4 -1.0 -1.0 -1.0 -1.0
 draw quad
 probe all rgba (1.0, 1.0, 1.0, 1.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float f;

float4 main() : sv_target
diff --git a/tests/hlsl/arithmetic-float-uniform.shader_test b/tests/hlsl/arithmetic-float-uniform.shader_test
index 4812d053a..61d369497 100644
--- a/tests/hlsl/arithmetic-float-uniform.shader_test
+++ b/tests/hlsl/arithmetic-float-uniform.shader_test
@@ -13,7 +13,7 @@ uniform 0 float4 5.0 15.0 0.0 0.0
 draw quad
 probe all rgba (20.0, -10.0, 75.0, 0.33333333) 1

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float2 a;

float4 main() : SV_TARGET
@@ -28,7 +28,7 @@ uniform 0 float4 5.0 15.0 0.0 0.0
 draw quad
 probe all rgba (5.0, 5.0, -5.0, 3.0) 1

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float2 a;

float4 main() : SV_TARGET
@@ -43,7 +43,7 @@ uniform 0 float4 42.0 5.0 0.0 0.0
 draw quad
 probe all rgba (2.0, -2.0, 2.0, -2.0) 16

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float2 a;

float4 main() : SV_TARGET
@@ -58,7 +58,7 @@ uniform 0 float4 45.0 5.0 0.0 0.0
 draw quad
 probe all rgba (0.0, 0.0, 0.0, 0.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float4 x, y;

float4 main() : sv_target
diff --git a/tests/hlsl/arithmetic-int-uniform.shader_test b/tests/hlsl/arithmetic-int-uniform.shader_test
index 7f5cdaaa6..d66efee62 100644
--- a/tests/hlsl/arithmetic-int-uniform.shader_test
+++ b/tests/hlsl/arithmetic-int-uniform.shader_test
@@ -1,4 +1,4 @@
-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float2 a;

float4 main() : SV_TARGET
@@ -13,7 +13,7 @@ uniform 0 float4 5.0 16.0 0.0 0.0
 draw quad
 probe all rgba (21.0, -11.0, 80.0, 0.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float2 a;

float4 main() : SV_TARGET
@@ -28,7 +28,7 @@ uniform 0 float4 5.0 16.0 0.0 0.0
 draw quad
 probe all rgba (5.0, 5.0, -5.0, 3.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float2 a;

float4 main() : SV_TARGET
@@ -43,7 +43,7 @@ uniform 0 float4 42.0 5.0 0.0 0.0
 draw quad
 probe all rgba (8.0, -8.0, -8.0, 8.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float2 a;

float4 main() : SV_TARGET
@@ -58,7 +58,7 @@ uniform 0 float4 42.0 5.0 0.0 0.0
 draw quad
 probe all rgba (2.0, -2.0, 2.0, -2.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float2 a;

float4 main() : SV_TARGET
@@ -73,7 +73,7 @@ uniform 0 float4 45.0 5.0 0.0 0.0
 draw quad
 probe all rgba (9.0, -9.0, -9.0, 9.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float2 a;

float4 main() : SV_TARGET
@@ -88,7 +88,7 @@ uniform 0 float4 45.0 5.0 0.0 0.0
 draw quad
 probe all rgba (0.0, 0.0, 0.0, 0.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 a;

float4 main() : SV_TARGET
@@ -101,7 +101,7 @@ uniform 0 float4 5.0 -7.0 0.0 -10.0
 todo(sm>=6) draw quad
 probe all rgba (5.0, 7.0, 0.0, 10.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 a;
 uniform float4 b;

diff --git a/tests/hlsl/arithmetic-int.shader_test b/tests/hlsl/arithmetic-int.shader_test
index f2044c42c..46b641811 100644
--- a/tests/hlsl/arithmetic-int.shader_test
+++ b/tests/hlsl/arithmetic-int.shader_test
@@ -77,6 +77,7 @@ draw quad
 probe all rgba (0.0, 0.0, 0.0, 0.0)

[pixel shader fail(sm<6)]
+// On SM1 this gives hr 0x88760b59.
 float4 main() : SV_TARGET
 {
     int x = 1;
@@ -90,6 +91,7 @@ draw quad
 probe all rgba (0.0, 0.0, 0.0, 0.0)

[pixel shader fail(sm<6)]
+// On SM1 this gives hr 0x88760b59.
 float4 main() : SV_TARGET
 {
     int x = 1;
diff --git a/tests/hlsl/ceil.shader_test b/tests/hlsl/ceil.shader_test
index 0ff9938b9..0d79c8686 100644
--- a/tests/hlsl/ceil.shader_test
+++ b/tests/hlsl/ceil.shader_test
@@ -21,7 +21,7 @@ uniform 0 float4 -0.5 6.5 7.5 3.4
 todo(sm>=6) draw quad
 probe all rgba (0.0, 7.0, 8.0, 4.0) 4

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
diff --git a/tests/hlsl/combined-samplers.shader_test b/tests/hlsl/combined-samplers.shader_test
index 235537594..4d743aecf 100644
--- a/tests/hlsl/combined-samplers.shader_test
+++ b/tests/hlsl/combined-samplers.shader_test
@@ -64,7 +64,7 @@ draw quad
 probe all rgba (10, 10, 10, 11)

-[pixel shader]
+[pixel shader todo(sm<4)]
 Texture2D tex;
 sampler sam[2];

@@ -111,7 +111,7 @@ probe all rgba (104, 104, 104, 111)

% Sampler arrays with components that have different usage dimensions are only forbidden in SM4 upwards.
 % However, tex2D and tex1D are considered the same dimension for these purposes.
-[pixel shader fail]
+[pixel shader fail(sm>=4)]
 sampler sam[2];

float4 main() : sv_target
diff --git a/tests/hlsl/conditional.shader_test b/tests/hlsl/conditional.shader_test
index 0a927a8fa..ec384b6c6 100644
--- a/tests/hlsl/conditional.shader_test
+++ b/tests/hlsl/conditional.shader_test
@@ -1,4 +1,4 @@
-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
@@ -17,7 +17,7 @@ uniform 0 float4 0.1 0.0 0.0 0.0
 draw quad
 probe all rgba (0.1, 0.2, 0.3, 0.4)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
@@ -29,7 +29,7 @@ float4 main() : sv_target
         return float4(0.9, 0.8, 0.7, 0.6);
 }

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
@@ -74,7 +74,7 @@ float main() : sv_target
 [require]
 shader model >= 3.0

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
diff --git a/tests/hlsl/discard.shader_test b/tests/hlsl/discard.shader_test
index cecb8c1fd..956cfe574 100644
--- a/tests/hlsl/discard.shader_test
+++ b/tests/hlsl/discard.shader_test
@@ -1,4 +1,4 @@
-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 x;

float4 main() : sv_target
diff --git a/tests/hlsl/distance.shader_test b/tests/hlsl/distance.shader_test
index 6527c218a..b4022aa5d 100644
--- a/tests/hlsl/distance.shader_test
+++ b/tests/hlsl/distance.shader_test
@@ -13,7 +13,7 @@ uniform 4 float4 2.0 -1.0 4.0 5.0
 todo(sm>=6) draw quad
 probe all rgba (7.483983, 7.483983, 7.483983, 7.483983) 1

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform int4 x;
 uniform int4 y;

diff --git a/tests/hlsl/expr-indexing.shader_test b/tests/hlsl/expr-indexing.shader_test
index c11fa6540..6e314f070 100644
--- a/tests/hlsl/expr-indexing.shader_test
+++ b/tests/hlsl/expr-indexing.shader_test
@@ -13,7 +13,7 @@ draw quad
 probe all rgba (8.0, 8.0, 8.0, 8.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float4 a, b;
 float i;

@@ -44,7 +44,7 @@ draw quad
 probe all rgba (3.0, 3.0, 3.0, 3.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float4 a;
 float i;

@@ -82,7 +82,7 @@ draw quad
 probe all rgba (4.0, 4.0, 4.0, 4.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float4 a;
 float i;

diff --git a/tests/hlsl/float-comparison.shader_test b/tests/hlsl/float-comparison.shader_test
index ee4b5eb7c..bc2e928ca 100644
--- a/tests/hlsl/float-comparison.shader_test
+++ b/tests/hlsl/float-comparison.shader_test
@@ -1,4 +1,4 @@
-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 f;

float4 main() : sv_target
diff --git a/tests/hlsl/floor.shader_test b/tests/hlsl/floor.shader_test
index e6562c4aa..9bc099e0a 100644
--- a/tests/hlsl/floor.shader_test
+++ b/tests/hlsl/floor.shader_test
@@ -21,7 +21,7 @@ uniform 0 float4 -0.5 6.5 7.5 3.4
 todo(sm>=6) draw quad
 probe all rgba (-1.0, 6.0, 7.0, 3.0) 4

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
diff --git a/tests/hlsl/fmod.shader_test b/tests/hlsl/fmod.shader_test
index 05518c7cd..70e993899 100644
--- a/tests/hlsl/fmod.shader_test
+++ b/tests/hlsl/fmod.shader_test
@@ -1,4 +1,4 @@
-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
@@ -14,7 +14,7 @@ uniform 0 float4 1.1 0.3 0.0 0.0
 todo(sm>=6) draw quad
 probe all rgba (0.2, 0.0, 0.0, 0.0) 4

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
diff --git a/tests/hlsl/for.shader_test b/tests/hlsl/for.shader_test
index 1392118b3..e30e083f3 100644
--- a/tests/hlsl/for.shader_test
+++ b/tests/hlsl/for.shader_test
@@ -4,7 +4,7 @@ void main(out float tex : texcoord, inout float4 pos : sv_position)
     tex = pos.x;
 }

-[pixel shader]
+[pixel shader todo(sm<4)]
 float4 main(float tex : texcoord) : sv_target
 {
     int i;
@@ -27,7 +27,7 @@ probe (  0, 0, 159, 480) rgba (10.0, 35.0, 0.0, 0.0)
 probe (161, 0, 479, 480) rgba (10.0, 38.0, 0.0, 0.0)
 probe (481, 0, 640, 480) rgba ( 5.0, 10.0, 0.0, 0.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float4 main(float tex : texcoord) : sv_target
 {
     int i;
diff --git a/tests/hlsl/function-return.shader_test b/tests/hlsl/function-return.shader_test
index be997d0c3..bf1f79372 100644
--- a/tests/hlsl/function-return.shader_test
+++ b/tests/hlsl/function-return.shader_test
@@ -32,8 +32,7 @@ float4 main() : sv_target
 draw quad
 probe all rgba (0.2, 0.1, 0.8, 0.5);

-[pixel shader]
-
+[pixel shader todo(sm<4)]
 uniform float f;

float func(out float o)
@@ -92,8 +91,7 @@ uniform 0 float 0.8
 todo(sm>=6) draw quad
 probe all rgba (0.8, 0.7, 0.4, 0.5) 1

-[pixel shader]
-
+[pixel shader todo(sm<4)]
 uniform float f;

float func(out float o)
@@ -145,8 +143,7 @@ uniform 0 float 0.9
 todo(sm>=6) draw quad
 probe all rgba (1.0, 0.9, 1.0, 0.6) 1

-[pixel shader]
-
+[pixel shader todo(sm<4)]
 float func(out float o)
 {
     o = 0.1;
@@ -187,8 +184,7 @@ float4 main() : sv_target
 draw quad
 probe all rgba (0.4, 0.3, 0.3, 0.9) 1

-[pixel shader]
-
+[pixel shader todo(sm<4)]
 uniform float f;

float func(out float o)
@@ -258,7 +254,7 @@ uniform 0 float 0.9
 todo(sm>=6) draw quad
 probe all rgba (0.4, 0.1, 0.7, 0.6) 1

-[pixel shader]
+[pixel shader todo(sm<4)]

uniform float4 f[3];

diff --git a/tests/hlsl/half.shader_test b/tests/hlsl/half.shader_test
index fe7074e45..2497ac36a 100644
--- a/tests/hlsl/half.shader_test
+++ b/tests/hlsl/half.shader_test
@@ -9,7 +9,7 @@ float4 main() : sv_target
 [require]
 options: backcompat

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform half h;

float4 main() : sv_target
diff --git a/tests/hlsl/hard-copy-prop.shader_test b/tests/hlsl/hard-copy-prop.shader_test
index 8d20425f1..623bfdc1d 100644
--- a/tests/hlsl/hard-copy-prop.shader_test
+++ b/tests/hlsl/hard-copy-prop.shader_test
@@ -1,4 +1,4 @@
-[pixel shader]
+[pixel shader todo(sm<4)]
 float cond;

float4 main() : sv_target
@@ -24,7 +24,7 @@ draw quad
 probe all rgba (-2.0, -2.0, -2.0, -2.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float cond;

float4 main() : sv_target
@@ -50,7 +50,7 @@ draw quad
 probe all rgba (20.0, 20.0, 20.0, 20.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float cond;

float4 main() : sv_target
@@ -66,6 +66,7 @@ float4 main() : sv_target

return float4(r, 0, 0);
 }
+
 [test]
 uniform 0 float 0.0
 draw quad
diff --git a/tests/hlsl/lit.shader_test b/tests/hlsl/lit.shader_test
index 5014c1ed0..0802c2718 100644
--- a/tests/hlsl/lit.shader_test
+++ b/tests/hlsl/lit.shader_test
@@ -1,4 +1,4 @@
-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
@@ -21,7 +21,7 @@ uniform 0 float4 1.2 2.0 3.0 0.0
 todo(sm>=6) draw quad
 probe all rgba (1.0, 1.2, 8.0, 1.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
diff --git a/tests/hlsl/loop.shader_test b/tests/hlsl/loop.shader_test
index 35a303595..9c142c1c3 100644
--- a/tests/hlsl/loop.shader_test
+++ b/tests/hlsl/loop.shader_test
@@ -1,6 +1,6 @@
 % TODO: dxcompiler emits no loops for any of these test shaders.

-[pixel shader]
+[pixel shader todo(sm<4)]
 float a;

float4 main() : sv_target
@@ -22,7 +22,7 @@ draw quad
 probe all rgba (50.0, 50.0, 50.0, 50.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float a;

float4 main() : sv_target
@@ -44,7 +44,7 @@ uniform 0 float 4.0
 draw quad
 probe all rgba (20.0, 20.0, 20.0, 20.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float a;

float4 main() : sv_target
@@ -73,7 +73,7 @@ uniform 0 float 4.0
 draw quad
 probe all rgba (409.1, 409.1, 409.1, 409.1)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float a;

float4 main() : sv_target
@@ -104,7 +104,7 @@ draw quad
 probe all rgba (410.1, 410.1, 410.1, 410.1)

% loop attribute by itself
-[pixel shader]
+[pixel shader todo(sm<4)]
 float4 main() : sv_target
 {
     float ret = 0.0;
@@ -121,7 +121,7 @@ float4 main() : sv_target
 todo(sm>=6) draw quad
 probe all rgba (10.0, 10.0, 10.0, 10.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float4 main() : sv_target
 {
     float ret = 0.0;
@@ -140,7 +140,7 @@ float4 main() : sv_target
 todo(sm>=6) draw quad
 probe all rgba (10.0, 10.0, 10.0, 10.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float4 main() : sv_target
 {
     float ret = 0.0;
diff --git a/tests/hlsl/matrix-indexing.shader_test b/tests/hlsl/matrix-indexing.shader_test
index 5024c8ed3..3200a7ee8 100644
--- a/tests/hlsl/matrix-indexing.shader_test
+++ b/tests/hlsl/matrix-indexing.shader_test
@@ -108,7 +108,7 @@ draw quad
 probe all rgba (3.0, 4.0, 50.0, 60.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float i;

float4 main() : sv_target
@@ -124,7 +124,7 @@ todo(sm>=6) draw quad
 probe all rgba (8, 9, 10, 11)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float i;

float4 main() : sv_target
diff --git a/tests/hlsl/non-const-indexing.shader_test b/tests/hlsl/non-const-indexing.shader_test
index 3a1e12acc..38509d1cb 100644
--- a/tests/hlsl/non-const-indexing.shader_test
+++ b/tests/hlsl/non-const-indexing.shader_test
@@ -1,4 +1,4 @@
-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 f[3];
 uniform float2 i;

@@ -25,7 +25,7 @@ draw quad
 probe all rgba (9.0, 10.0, 11.0, 12.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float i;

float4 main() : SV_TARGET
@@ -49,7 +49,7 @@ todo(sm>=6) draw quad
 probe all rgba (14.0, 14.0, 14.0, 14.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float i;

float4 main() : sv_target
@@ -65,7 +65,7 @@ todo(sm>=6) draw quad
 todo(sm>=6) probe all rgba (3, 3, 3, 3)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float i;

float4 main() : SV_TARGET
@@ -90,7 +90,7 @@ todo(sm>=6) draw quad
 probe all rgba (24.0, 0.0, 21.0, 1.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float2 i;

float4 main() : sv_target
@@ -115,7 +115,7 @@ todo(sm>=6) draw quad
 todo(sm>=6) probe all rgba (9.0, 10.0, 11.0, 12.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 float4 a;

float4 main() : sv_target
diff --git a/tests/hlsl/return.shader_test b/tests/hlsl/return.shader_test
index fbea07926..2970aeaef 100644
--- a/tests/hlsl/return.shader_test
+++ b/tests/hlsl/return.shader_test
@@ -25,8 +25,7 @@ void main(out float4 ret : sv_target)
 draw quad
 probe all rgba (0.1, 0.2, 0.3, 0.4)

-[pixel shader]
-
+[pixel shader todo(sm<4)]
 uniform float f;

float4 main() : sv_target
@@ -44,8 +43,7 @@ uniform 0 float 0.8
 draw quad
 probe all rgba (0.5, 0.6, 0.7, 0.8)

-[pixel shader]
-
+[pixel shader todo(sm<4)]
 uniform float f;

void main(out float4 ret : sv_target)
@@ -71,8 +69,7 @@ uniform 0 float 0.8
 draw quad
 probe all rgba (0.1, 0.2, 0.3, 0.4)

-[pixel shader]
-
+[pixel shader todo(sm<4)]
 uniform float f;

void main(out float4 ret : sv_target)
@@ -101,8 +98,7 @@ uniform 0 float 0.9
 todo(sm>=6) draw quad
 probe all rgba (0.5, 0.6, 0.7, 0.8) 1

-[pixel shader]
-
+[pixel shader todo(sm<4)]
 uniform float f;

void main(out float4 ret : sv_target)
@@ -128,8 +124,7 @@ uniform 0 float 0.9
 todo(sm>=6) draw quad
 probe all rgba (0.4, 0.5, 0.6, 0.7) 1

-[pixel shader]
-
+[pixel shader todo(sm<4)]
 void main(out float4 ret : sv_target)
 {
     ret = float4(0.1, 0.2, 0.3, 0.4);
@@ -146,8 +141,7 @@ void main(out float4 ret : sv_target)
 draw quad
 probe all rgba (0.2, 0.4, 0.6, 0.8)

-[pixel shader]
-
+[pixel shader todo(sm<4)]
 uniform float f;

void main(out float4 ret : sv_target)
@@ -185,8 +179,7 @@ uniform 0 float 0.9
 todo(sm>=6) draw quad
 probe all rgba (0.9, 0.9, 0.9, 0.9) 1

-[pixel shader]
-
+[pixel shader todo(sm<4)]
 uniform float f;

void main(out float4 ret : sv_target)
@@ -217,7 +210,7 @@ uniform 0 float 0.8
 draw quad
 probe all rgba (0.5, 0.5, 0.5, 0.5)

-[pixel shader]
+[pixel shader todo(sm<4)]

uniform float4 f[3];

diff --git a/tests/hlsl/round.shader_test b/tests/hlsl/round.shader_test
index 2a32b3ec2..51e273634 100644
--- a/tests/hlsl/round.shader_test
+++ b/tests/hlsl/round.shader_test
@@ -13,7 +13,7 @@ probe all rgba (0.0, -7.0, 8.0, 3.0) 4

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
@@ -31,7 +31,7 @@ probe all rgba (-7.0, 8.0, 0.0, 3.0) 4

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
diff --git a/tests/hlsl/sample-bias.shader_test b/tests/hlsl/sample-bias.shader_test
index e56945d06..3f5b28095 100644
--- a/tests/hlsl/sample-bias.shader_test
+++ b/tests/hlsl/sample-bias.shader_test
@@ -17,7 +17,7 @@ void main(out float2 tex : texcoord, inout float4 pos : sv_position)
     tex = pos.xy;
 }

-[pixel shader]
+[pixel shader todo(sm<4)]
 sampler s;
 Texture2D t;
 uniform float bias;
diff --git a/tests/hlsl/sample-level.shader_test b/tests/hlsl/sample-level.shader_test
index 8b2890ff7..a0fd5fdcc 100644
--- a/tests/hlsl/sample-level.shader_test
+++ b/tests/hlsl/sample-level.shader_test
@@ -14,7 +14,7 @@ levels 2

0.0 0.0 1.0 0.0

-[pixel shader]
+[pixel shader todo(sm<4)]
 sampler s;
 Texture2D t;
 uniform float level;
diff --git a/tests/hlsl/sampler.shader_test b/tests/hlsl/sampler.shader_test
index caff7b2fa..d857c4b18 100644
--- a/tests/hlsl/sampler.shader_test
+++ b/tests/hlsl/sampler.shader_test
@@ -7,7 +7,7 @@ size (2, 2)
 0.0 0.0 0.0 0.0     0.0 0.0 0.0 0.0
 0.0 0.0 0.0 0.0     1.0 0.0 1.0 0.0

-[pixel shader]
+[pixel shader todo(sm<4)]
 sampler s;
 Texture2D t;

@@ -20,7 +20,7 @@ float4 main() : sv_target
 todo(sm>=6) draw quad
 probe all rgba (0.25, 0, 0.25, 0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 SamplerState s;
 Texture2D t;

diff --git a/tests/hlsl/saturate.shader_test b/tests/hlsl/saturate.shader_test
index 2ed83cf66..56f84c773 100644
--- a/tests/hlsl/saturate.shader_test
+++ b/tests/hlsl/saturate.shader_test
@@ -11,7 +11,7 @@ uniform 0 float4 0.7 -0.1 0.0 0.0
 todo(sm>=6) draw quad
 probe all rgba (0.7, 0.0, 1.0, 0.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
diff --git a/tests/hlsl/sign.shader_test b/tests/hlsl/sign.shader_test
index 5d8b43168..f676bd41f 100644
--- a/tests/hlsl/sign.shader_test
+++ b/tests/hlsl/sign.shader_test
@@ -1,4 +1,4 @@
-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float f;

float4 main() : sv_target
@@ -17,7 +17,7 @@ uniform 0 float4 0.0 0.0 0.0 0.0
 draw quad
 probe all rgba (0.0, 0.0, 0.0, 0.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 f;

float4 main() : sv_target
@@ -30,7 +30,7 @@ uniform 0 float4 1.0 2.0 3.0 4.0
 draw quad
 probe all rgba (1.0, 1.0, 1.0, 1.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float2x2 f;

float4 main() : sv_target
@@ -48,7 +48,7 @@ probe all rgba (1.0, 1.0, 1.0, 1.0)
 % SM1-3 doesn't support integral types
 shader model >= 4.0

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform int f;

float4 main() : sv_target
@@ -67,7 +67,7 @@ uniform 0 int4 0 0 0 0
 draw quad
 probe all rgba (0, 0, 0, 0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform int4 f;

float4 main() : sv_target
@@ -80,7 +80,7 @@ uniform 0 int4 1 2 3 4
 draw quad
 probe all rgba (1, 1, 1, 1)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform int2x2 f;

float4 main() : sv_target
diff --git a/tests/hlsl/smoothstep.shader_test b/tests/hlsl/smoothstep.shader_test
index 971f6d5d8..38e1432ca 100644
--- a/tests/hlsl/smoothstep.shader_test
+++ b/tests/hlsl/smoothstep.shader_test
@@ -109,7 +109,7 @@ draw quad
 probe all rgba (0.028, 0.104, 0.216, 0.352) 6

-[pixel shader]
+[pixel shader todo(sm<4)]
 // 4 division by zero warnings.
 // Only test compilation because result is implementation-dependent.
 float4 main() : sv_target
diff --git a/tests/hlsl/static-initializer.shader_test b/tests/hlsl/static-initializer.shader_test
index 217444308..aec1dac1e 100644
--- a/tests/hlsl/static-initializer.shader_test
+++ b/tests/hlsl/static-initializer.shader_test
@@ -17,6 +17,7 @@ probe all rgba (0.8, 0.0, 0.0, 0.0)

[pixel shader fail(sm<6)]
+// On SM1 this gives hr 0x88760b59.
 static uint i;

float4 main() : sv_target
@@ -135,7 +136,7 @@ float4 main(Texture2D tex2) : sv_target
 }

-[pixel shader]
+[pixel shader todo(sm<4)]
 Texture2D real_tex;
 static Texture2D tex = real_tex;
 sampler sam;
@@ -150,7 +151,7 @@ todo(sm>=6) draw quad
 probe all rgba (1, 2, 3, 4)

-[pixel shader]
+[pixel shader todo(sm<4)]
 Texture2D real_tex;
 static Texture2D tex;
 sampler sam;
diff --git a/tests/hlsl/step.shader_test b/tests/hlsl/step.shader_test
index e201e15f9..546505fbd 100644
--- a/tests/hlsl/step.shader_test
+++ b/tests/hlsl/step.shader_test
@@ -1,4 +1,4 @@
-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 f, p;

float4 main() : sv_target
diff --git a/tests/hlsl/trigonometry.shader_test b/tests/hlsl/trigonometry.shader_test
index f52d01dea..5736696f6 100644
--- a/tests/hlsl/trigonometry.shader_test
+++ b/tests/hlsl/trigonometry.shader_test
@@ -4,7 +4,7 @@ void main(out float tex : texcoord, inout float4 pos : sv_position)
     tex = (pos.x + 1) * 320;
 }

-[pixel shader]
+[pixel shader todo(sm<4)]
 float4 main(float tex : texcoord) : sv_target
 {
     tex = floor(tex + 0.25);
@@ -31,7 +31,7 @@ probe (14, 0) rgba ( 0.99060736,  0.13673722,  7.24460662, 0.0) 1024
 probe (15, 0) rgba ( 0.65028784, -0.75968791, -0.85599340, 0.0) 1024

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 a;

float4 main() : sv_target
@@ -45,7 +45,7 @@ todo(sm>=6) draw quad
 probe all rgba (0.0, 500.0, 500.0, 0.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 a;

float4 main() : sv_target
@@ -59,7 +59,7 @@ todo(sm>=6) draw quad
 probe all rgba (1000.0, 707.0, -0.0, -707.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 a;

float4 main() : sv_target
diff --git a/tests/hlsl/trunc.shader_test b/tests/hlsl/trunc.shader_test
index a359cd03a..85631f25e 100644
--- a/tests/hlsl/trunc.shader_test
+++ b/tests/hlsl/trunc.shader_test
@@ -1,4 +1,4 @@
-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
@@ -14,7 +14,7 @@ uniform 0 float4 -1.5 6.5 7.5 3.4
 todo(sm>=6) draw quad
 probe all rgba (-1.0, 6.0, 7.0, 3.0)

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float4 u;

float4 main() : sv_target
diff --git a/tests/hlsl/vector-indexing-uniform.shader_test b/tests/hlsl/vector-indexing-uniform.shader_test
index 3501f3af7..63e7ec054 100644
--- a/tests/hlsl/vector-indexing-uniform.shader_test
+++ b/tests/hlsl/vector-indexing-uniform.shader_test
@@ -1,6 +1,6 @@
 % Use a uniform to prevent the compiler from optimizing.

-[pixel shader]
+[pixel shader todo(sm<4)]
 uniform float i;
 float4 main() : SV_TARGET
 {
diff --git a/tests/shader_runner.c b/tests/shader_runner.c
index ec6aac427..c4b7dee1d 100644
--- a/tests/shader_runner.c
+++ b/tests/shader_runner.c
@@ -955,7 +955,7 @@ unsigned int get_vb_stride(const struct shader_runner *runner, unsigned int slot

static HRESULT map_unidentified_hrs(HRESULT hr)
 {
-    if (hr == 0x80010064)
+    if (hr == 0x80010064 || hr == 0x88760b59)
     {
         trace("Mapping unidentified hr %#x as %#x.\n", hr, E_FAIL);
         return E_FAIL;
@@ -1101,8 +1101,8 @@ static void compile_shader(struct shader_runner *runner, IDxcCompiler3 *dxc_comp

static const char *const shader_models[] =
     {
-        [SHADER_MODEL_2_0] = "4_0",
-        [SHADER_MODEL_3_0] = "4_0",
+        [SHADER_MODEL_2_0] = "2_0",
+        [SHADER_MODEL_3_0] = "3_0",
         [SHADER_MODEL_4_0] = "4_0",
         [SHADER_MODEL_4_1] = "4_1",
         [SHADER_MODEL_5_0] = "5_0",
@@ -1200,7 +1200,7 @@ static enum parse_state read_shader_directive(struct shader_runner *runner, enum
 }

void run_shader_tests(struct shader_runner *runner, const struct shader_runner_ops *ops, void *dxc_compiler,
-        enum shader_model minimum_shader_model, enum shader_model maximum_shader_model)
+        enum shader_model minimum_shader_model, enum shader_model maximum_shader_model, bool skip_execution)
 {
     size_t shader_source_size = 0, shader_source_len = 0;
     struct resource_params current_resource;
@@ -1223,6 +1223,7 @@ void run_shader_tests(struct shader_runner *runner, const struct shader_runner_o
     runner->ops = ops;
     runner->minimum_shader_model = minimum_shader_model;
     runner->maximum_shader_model = maximum_shader_model;
+    runner->skip_execution = skip_execution;

for (;;)
     {
@@ -1571,7 +1572,8 @@ void run_shader_tests(struct shader_runner *runner, const struct shader_runner_o
                 case STATE_TEST:
                     /* Compilation which fails with dxcompiler is not 'todo', therefore the tests are
                      * not 'todo' either. They cannot run, so skip them entirely. */
-                    if (!skip_tests && SUCCEEDED(expect_hr))
+                    printf("Skip execution? %u\n", runner->skip_execution);
+                    if (!skip_tests && !runner->skip_execution && SUCCEEDED(expect_hr))
                         parse_test_directive(runner, line);
                     break;
             }
@@ -1690,13 +1692,13 @@ START_TEST(shader_runner)
 #if defined(VKD3D_CROSSTEST)
     trace("Running tests from a Windows cross build\n");

-    trace("Compiling shaders with d3dcompiler_47.dll and executing with d3d9.dll\n");
+    trace("Compiling SM1 shaders with d3dcompiler_47.dll and executing with d3d9.dll\n");
     run_shader_tests_d3d9();

-    trace("Compiling shaders with d3dcompiler_47.dll and executing with d3d11.dll\n");
+    trace("Compiling SM4 shaders with d3dcompiler_47.dll and executing with d3d11.dll\n");
     run_shader_tests_d3d11();

-    trace("Compiling shaders with d3dcompiler_47.dll and executing with d3d12.dll\n");
+    trace("Compiling SM4 shaders with d3dcompiler_47.dll and executing with d3d12.dll\n");
     run_shader_tests_d3d12(NULL, SHADER_MODEL_4_0, SHADER_MODEL_5_1);

print_dll_version("d3dcompiler_47.dll");
@@ -1707,18 +1709,18 @@ START_TEST(shader_runner)
 #elif defined(_WIN32)
     trace("Running tests from a Windows non-cross build\n");

-    trace("Compiling shaders with vkd3d-shader and executing with d3d9.dll\n");
+    trace("Compiling SM1 shaders with vkd3d-shader and executing with d3d9.dll\n");
     run_shader_tests_d3d9();

-    trace("Compiling shaders with vkd3d-shader and executing with d3d11.dll\n");
+    trace("Compiling SM4 shaders with vkd3d-shader and executing with d3d11.dll\n");
     run_shader_tests_d3d11();

-    trace("Compiling shaders with vkd3d-shader and executing with vkd3d\n");
+    trace("Compiling SM4 shaders with vkd3d-shader and executing with vkd3d\n");
     run_shader_tests_d3d12(NULL, SHADER_MODEL_4_0, SHADER_MODEL_5_1);

if ((dxc_compiler = dxcompiler_create()))
     {
-        trace("Compiling shaders with dxcompiler and executing with vkd3d\n");
+        trace("Compiling SM6 shaders with dxcompiler and executing with vkd3d\n");
         run_shader_tests_d3d12(dxc_compiler, SHADER_MODEL_6_0, SHADER_MODEL_6_0);
         IDxcCompiler3_Release(dxc_compiler);
         print_dll_version(SONAME_LIBDXCOMPILER);
@@ -1734,15 +1736,18 @@ START_TEST(shader_runner)
     run_shader_tests_gl();
 # endif

-    trace("Compiling shaders with vkd3d-shader and executing with Vulkan\n");
-    run_shader_tests_vulkan();
+    trace("Compiling SM1 shaders with vkd3d-shader\n");
+    run_shader_tests_vulkan(SHADER_MODEL_2_0, SHADER_MODEL_3_0, true);
+
+    trace("Compiling SM4 shaders with vkd3d-shader and executing with Vulkan\n");
+    run_shader_tests_vulkan(SHADER_MODEL_4_0, SHADER_MODEL_5_1, false);

-    trace("Compiling shaders with vkd3d-shader and executing with vkd3d\n");
+    trace("Compiling SM4 shaders with vkd3d-shader and executing with vkd3d\n");
     run_shader_tests_d3d12(NULL, SHADER_MODEL_4_0, SHADER_MODEL_5_1);

if ((dxc_compiler = dxcompiler_create()))
     {
-        trace("Compiling shaders with dxcompiler and executing with vkd3d\n");
+        trace("Compiling SM6 shaders with dxcompiler and executing with vkd3d\n");
         run_shader_tests_d3d12(dxc_compiler, SHADER_MODEL_6_0, SHADER_MODEL_6_0);
         IDxcCompiler3_Release(dxc_compiler);
     }
diff --git a/tests/shader_runner.h b/tests/shader_runner.h
index a4f0d2fd5..a4fa98609 100644
--- a/tests/shader_runner.h
+++ b/tests/shader_runner.h
@@ -115,6 +115,7 @@ struct shader_runner
     const struct shader_runner_ops *ops;

bool is_todo;
+    bool skip_execution;

char *vs_source;
     char *ps_source;
@@ -168,14 +169,15 @@ struct sampler *shader_runner_get_sampler(struct shader_runner *runner, unsigned
 struct resource *shader_runner_get_resource(struct shader_runner *runner, enum resource_type type, unsigned int slot);

void run_shader_tests(struct shader_runner *runner, const struct shader_runner_ops *ops, void *dxc_compiler,
-        enum shader_model minimum_shader_model, enum shader_model maximum_shader_model);
+        enum shader_model minimum_shader_model, enum shader_model maximum_shader_model, bool skip_execution);

#ifdef _WIN32
 void run_shader_tests_d3d9(void);
 void run_shader_tests_d3d11(void);
 #else
 void run_shader_tests_gl(void);
-void run_shader_tests_vulkan(void);
+void run_shader_tests_vulkan(enum shader_model minimum_shader_model, enum shader_model maximum_shader_model,
+        bool skip_execution);
 #endif
 void run_shader_tests_d3d12(void *dxc_compiler, enum shader_model minimum_shader_model,
         enum shader_model maximum_shader_model);
diff --git a/tests/shader_runner_d3d11.c b/tests/shader_runner_d3d11.c
index abe455203..4aad97966 100644
--- a/tests/shader_runner_d3d11.c
+++ b/tests/shader_runner_d3d11.c
@@ -747,7 +747,7 @@ void run_shader_tests_d3d11(void)
         init_adapter_info();
         if (init_test_context(&runner))
         {
-            run_shader_tests(&runner.r, &d3d11_runner_ops, NULL, SHADER_MODEL_4_0, SHADER_MODEL_5_1);
+            run_shader_tests(&runner.r, &d3d11_runner_ops, NULL, SHADER_MODEL_4_0, SHADER_MODEL_5_1, false);
             destroy_test_context(&runner);
         }
     }
diff --git a/tests/shader_runner_d3d12.c b/tests/shader_runner_d3d12.c
index a05dfd059..dc531482f 100644
--- a/tests/shader_runner_d3d12.c
+++ b/tests/shader_runner_d3d12.c
@@ -602,7 +602,7 @@ void run_shader_tests_d3d12(void *dxc_compiler, enum shader_model minimum_shader
             runner.compute_allocator, NULL, &IID_ID3D12GraphicsCommandList, (void **)&runner.compute_list);
     ok(hr == S_OK, "Failed to create command list, hr %#x.\n", hr);

-    run_shader_tests(&runner.r, &d3d12_runner_ops, dxc_compiler, minimum_shader_model, maximum_shader_model);
+    run_shader_tests(&runner.r, &d3d12_runner_ops, dxc_compiler, minimum_shader_model, maximum_shader_model, false);

ID3D12GraphicsCommandList_Release(runner.compute_list);
     ID3D12CommandAllocator_Release(runner.compute_allocator);
diff --git a/tests/shader_runner_d3d9.c b/tests/shader_runner_d3d9.c
index 0b33c1c53..8112d465c 100644
--- a/tests/shader_runner_d3d9.c
+++ b/tests/shader_runner_d3d9.c
@@ -541,7 +541,7 @@ void run_shader_tests_d3d9(void)

init_adapter_info();
         init_test_context(&runner);
-        run_shader_tests(&runner.r, &d3d9_runner_ops, NULL, SHADER_MODEL_2_0, SHADER_MODEL_3_0);
+        run_shader_tests(&runner.r, &d3d9_runner_ops, NULL, SHADER_MODEL_2_0, SHADER_MODEL_3_0, false);
         destroy_test_context(&runner);
     }
     FreeLibrary(d3d9_module);
diff --git a/tests/shader_runner_gl.c b/tests/shader_runner_gl.c
index 8db0ed68d..acbf09839 100644
--- a/tests/shader_runner_gl.c
+++ b/tests/shader_runner_gl.c
@@ -996,7 +996,7 @@ void run_shader_tests_gl(void)
     vkd3d_test_name = "shader_runner_gl";
     if (!gl_runner_init(&runner))
         goto done;
-    run_shader_tests(&runner.r, &gl_runner_ops, NULL, SHADER_MODEL_4_0, SHADER_MODEL_5_1);
+    run_shader_tests(&runner.r, &gl_runner_ops, NULL, SHADER_MODEL_4_0, SHADER_MODEL_5_1, false);
     gl_runner_cleanup(&runner);
 done:
     vkd3d_test_name = test_name;
diff --git a/tests/shader_runner_vulkan.c b/tests/shader_runner_vulkan.c
index f89b4d624..d4969780c 100644
--- a/tests/shader_runner_vulkan.c
+++ b/tests/shader_runner_vulkan.c
@@ -395,8 +395,8 @@ static bool compile_shader(const struct vulkan_shader_runner *runner, const char

static const char *const shader_models[] =
     {
-        [SHADER_MODEL_2_0] = "4_0",
-        [SHADER_MODEL_3_0] = "4_0",
+        [SHADER_MODEL_2_0] = "2_0",
+        [SHADER_MODEL_3_0] = "3_0",
         [SHADER_MODEL_4_0] = "4_0",
         [SHADER_MODEL_4_1] = "4_1",
         [SHADER_MODEL_5_0] = "5_0",
@@ -407,7 +407,11 @@ static bool compile_shader(const struct vulkan_shader_runner *runner, const char
     info.source.code = source;
     info.source.size = strlen(source);
     info.source_type = VKD3D_SHADER_SOURCE_HLSL;
-    info.target_type = VKD3D_SHADER_TARGET_DXBC_TPF;
+    if (runner->r.minimum_shader_model < SHADER_MODEL_4_0)
+        info.target_type = VKD3D_SHADER_TARGET_D3D_BYTECODE;
+    else
+        info.target_type = VKD3D_SHADER_TARGET_DXBC_TPF;
+
     info.log_level = VKD3D_SHADER_LOG_WARNING;

info.options = options;
@@ -1377,14 +1381,16 @@ static void cleanup_vulkan_runner(struct vulkan_shader_runner *runner)
     VK_CALL(vkDestroyInstance(runner->instance, NULL));
 }

-void run_shader_tests_vulkan(void)
+void run_shader_tests_vulkan(enum shader_model minimum_shader_model, enum shader_model maximum_shader_model,
+        bool skip_execution)
 {
     struct vulkan_shader_runner runner = {0};

if (!init_vulkan_runner(&runner))
         return;

-    run_shader_tests(&runner.r, &vulkan_runner_ops, NULL, SHADER_MODEL_2_0, SHADER_MODEL_5_1);
+    run_shader_tests(&runner.r, &vulkan_runner_ops, NULL, minimum_shader_model, maximum_shader_model,
+            skip_execution);

cleanup_vulkan_runner(&runner);
 }

[Francisco Casas (＠fcasas)]

Francisco Casas (@fcasas) commented about tests/shader_runner.c:
>              case STATE_TEST:
>                  /* Compilation which fails with dxcompiler is not 'todo', therefore the tests are
>                   * not 'todo' either. They cannot run, so skip them entirely. */
> -
>                if (!skip_tests && SUCCEEDED(expect_hr))
> -
>                printf("Skip execution? %u\n", runner->skip_execution);

You probably forgot to remove this, me.

[Francisco Casas (＠fcasas)]

For the record, it is possible to squash this [commit](https://gitlab.winehq.org/fcasas/vkd3d/-/commit/4fd2ba66b1783c7e346f520b1ea3...) in 3/3 if we want to avoid the `skip_execution` flag and add `todo(sm<4)` qualifiers instead.

The problem is that these would be almost everywhere before we add SM1 execution support, and also, until we properly parse the d3dbc, they have to be in the `draw` directives instead of the `quad` directives so I am not sure is a good idea.

[Henri Verbeet (＠hverbeet)]

> So if we have tests that are todo on both sm<4 and sm>=6 we can write,
> e.g.:
>
>     todo(sm<4) todo(sm>=6) draw draw
>
> OTOH if we were to have tests that are todo only in sm>=4 and sm<6 we
> can write:
>
>     todo(sm>=4,sm<6) draw quad
>
> Effectively, multiple arguments in a single todo() are like an AND and
> multiple todo()-s behave like an OR.

I'd like to hear from others as well, but that syntax doesn't seem ideal. Using "," for "and" in particular seems prone to confusion. Some possible alternatives:

- "todo(sm<4,sm>=6)" and "todo(sm<4+sm>=6)"
  - "todo(sm<4 | sm>=6)" and "todo(sm<4 & sm>=6)"
  - "todo(sm<4 ∪ sm>=6)" and "todo(sm<4 ∩ sm>=6)"
>  static HRESULT map_unidentified_hrs(HRESULT hr)
>  {
> -    if (hr == 0x80010064)
> +    if (hr == 0x80010064 || hr == 0x88760b59)

0x88760b59 is  D3DXERR_INVALIDDATA.
> @@ -1690,13 +1692,13 @@ START_TEST(shader_runner)
>  #if defined(VKD3D_CROSSTEST)
>      trace("Running tests from a Windows cross build\n");
>
> -    trace("Compiling shaders with d3dcompiler_47.dll and executing with d3d9.dll\n");
> +    trace("Compiling SM1 shaders with d3dcompiler_47.dll and executing with d3d9.dll\n");
>      run_shader_tests_d3d9();
>
> -    trace("Compiling shaders with d3dcompiler_47.dll and executing with d3d11.dll\n");
> +    trace("Compiling SM4 shaders with d3dcompiler_47.dll and executing with d3d11.dll\n");
>      run_shader_tests_d3d11();
>
> -    trace("Compiling shaders with d3dcompiler_47.dll and executing with d3d12.dll\n");
> +    trace("Compiling SM4 shaders with d3dcompiler_47.dll and executing with d3d12.dll\n");
>      run_shader_tests_d3d12(NULL, SHADER_MODEL_4_0, SHADER_MODEL_5_1);
>
>      print_dll_version("d3dcompiler_47.dll");

These traces (and others) probably make more sense inside their corresponding run_shader_tests_*() functions. Relatedly, ideally we'd only have a single invocation of run_shader_tests_vulkan() and run_shader_tests_d3d12(), which would then take care of invoking run_shader_tests() multiple times themselves.
> @@ -407,7 +407,11 @@ static bool compile_shader(const struct vulkan_shader_runner *runner, const char
>      info.source.code = source;
>      info.source.size = strlen(source);
>      info.source_type = VKD3D_SHADER_SOURCE_HLSL;
> -    info.target_type = VKD3D_SHADER_TARGET_DXBC_TPF;
> +    if (runner->r.minimum_shader_model < SHADER_MODEL_4_0)
> +        info.target_type = VKD3D_SHADER_TARGET_D3D_BYTECODE;
> +    else
> +        info.target_type = VKD3D_SHADER_TARGET_DXBC_TPF;
> +
>      info.log_level = VKD3D_SHADER_LOG_WARNING;
>
>      info.options = options;

You should also change the "info.source_type = VKD3D_SHADER_SOURCE_DXBC_TPF;" line a bit further down to something like "info.source_type = info.target_type;".
> > For the record, it is possible to squash this [commit](https://gitlab.winehq.org/fcasas/vkd3d/-/commit/4fd2ba66b1783c7e346f520b1ea3...) in 3/3 if we want to avoid the `skip_execution` flag and add `todo(sm<4)` qualifiers instead.
>
> The problem is that these would be almost everywhere before we add SM1 execution support, and also, until we properly parse the d3dbc, they have to be in the `draw` directives instead of the `probe` directives so I am not sure is a good idea.

It doesn't seem too bad to me. For comparison, see commit 57280673e51f6b60487e91369bd57213a8243a56.

[Zebediah Figura (＠zfigura)]

I've just submitted 518 which may serve as a better basis for this merge request.

2.x isn't supported yet (though it should only take a couple patches which I've already written), but we can start with 3.0.