[PATCH vkd3d 1/4] vkd3d-compiler: Rename the 'spirv' local variable to 'output_code'.

[Zebediah Figura]

Signed-off-by: Zebediah Figura <zfigura(a)codeweavers.com>
---
 programs/vkd3d-compiler/main.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/programs/vkd3d-compiler/main.c b/programs/vkd3d-compiler/main.c
index 7a1a5b0b..bcca04c3 100644
--- a/programs/vkd3d-compiler/main.c
+++ b/programs/vkd3d-compiler/main.c
@@ -586,7 +586,7 @@ int main(int argc, char **argv)
     struct vkd3d_shader_hlsl_source_info hlsl_source_info = {0};
     bool close_input = false, close_output = false;
     struct vkd3d_shader_compile_info info;
-    struct vkd3d_shader_code spirv;
+    struct vkd3d_shader_code output_code;
     struct options options;
     FILE *input, *output;
     char *messages;
@@ -663,7 +663,7 @@ int main(int argc, char **argv)
         goto done;
     }
 
-    ret = vkd3d_shader_compile(&info, &spirv, &messages);
+    ret = vkd3d_shader_compile(&info, &output_code, &messages);
     if (messages)
         fputs(messages, stderr);
     vkd3d_shader_free_messages(messages);
@@ -674,15 +674,15 @@ int main(int argc, char **argv)
         goto done;
     }
 
-    if (!write_shader(&spirv, output))
+    if (!write_shader(&output_code, output))
     {
         fprintf(stderr, "Failed to write output shader.\n");
-        vkd3d_shader_free_shader_code(&spirv);
+        vkd3d_shader_free_shader_code(&output_code);
         goto done;
     }
 
     fail = 0;
-    vkd3d_shader_free_shader_code(&spirv);
+    vkd3d_shader_free_shader_code(&output_code);
 done:
     if (close_output)
         fclose(output);
-- 
2.29.2

[PATCH vkd3d 2/4] vkd3d-compiler: Also pass the entry point as part of the SPIR-V target information.

[Zebediah Figura]

Signed-off-by: Zebediah Figura <zfigura(a)codeweavers.com>
---
 programs/vkd3d-compiler/main.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/programs/vkd3d-compiler/main.c b/programs/vkd3d-compiler/main.c
index bcca04c3..3eee4bd2 100644
--- a/programs/vkd3d-compiler/main.c
+++ b/programs/vkd3d-compiler/main.c
@@ -583,6 +583,7 @@ static bool has_colour(FILE *f)
 
 int main(int argc, char **argv)
 {
+    struct vkd3d_shader_spirv_target_info spirv_target_info = {0};
     struct vkd3d_shader_hlsl_source_info hlsl_source_info = {0};
     bool close_input = false, close_output = false;
     struct vkd3d_shader_compile_info info;
@@ -654,9 +655,14 @@ int main(int argc, char **argv)
     info.source_name = options.filename;
 
     hlsl_source_info.type = VKD3D_SHADER_STRUCTURE_TYPE_HLSL_SOURCE_INFO;
+    hlsl_source_info.next = &spirv_target_info;
     hlsl_source_info.profile = options.profile;
     hlsl_source_info.entry_point = options.entry_point;
 
+    spirv_target_info.type = VKD3D_SHADER_STRUCTURE_TYPE_SPIRV_TARGET_INFO;
+    spirv_target_info.entry_point = options.entry_point;
+    spirv_target_info.environment = VKD3D_SHADER_SPIRV_ENVIRONMENT_VULKAN_1_0;
+
     if (!read_shader(&info.source, input))
     {
         fprintf(stderr, "Failed to read input shader.\n");
-- 
2.29.2

[PATCH vkd3d 2/4] vkd3d-compiler: Also pass the entry point as part of the SPIR-V target information.

[Marvin]

Hi,

While running your changed tests, I think I found new failures.
Being a bot and all I'm not very good at pattern recognition, so I might be
wrong, but could you please double-check?

Full results can be found at:
https://testbot.winehq.org/JobDetails.pl?Key=82560

Your paranoid android.


=== debiant (build log) ===

Task: Patch failed to apply

=== debiant (build log) ===

Task: Patch failed to apply

[PATCH vkd3d 2/4] vkd3d-compiler: Also pass the entry point as part of the SPIR-V target information.

[Henri Verbeet]

Signed-off-by: Henri Verbeet <hverbeet(a)codeweavers.com>

[PATCH vkd3d 3/4] vkd3d-compiler: Add support for running the HLSL preprocessor.

[Zebediah Figura]

Signed-off-by: Zebediah Figura <zfigura(a)codeweavers.com>
---
 programs/vkd3d-compiler/main.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/programs/vkd3d-compiler/main.c b/programs/vkd3d-compiler/main.c
index 3eee4bd2..cec2ebb1 100644
--- a/programs/vkd3d-compiler/main.c
+++ b/programs/vkd3d-compiler/main.c
@@ -163,6 +163,7 @@ static void print_usage(const char *program_name)
         "                        Valid values are 'buffer-texture' (default) and\n"
         "                        'storage-buffer'.\n"
         "  -e, --entry=<name>    Use <name> as the entry point (default is \"main\").\n"
+        "  -E                    Preprocess the source code instead of compiling it.\n"
         "  -o, --output=<file>   Write the output to <file>. If <file> is '-' or no\n"
         "                        output file is specified, output will be written to\n"
         "                        standard output.\n"
@@ -196,6 +197,7 @@ struct options
     const char *profile;
     enum vkd3d_shader_source_type source_type;
     enum vkd3d_shader_target_type target_type;
+    bool preprocess_only;
     bool print_version;
     bool print_source_types;
     bool print_target_types;
@@ -402,7 +404,7 @@ static bool parse_command_line(int argc, char **argv, struct options *options)
 
     for (;;)
     {
-        if ((option = getopt_long(argc, argv, "b:e:ho:p:Vx:", long_options, NULL)) == -1)
+        if ((option = getopt_long(argc, argv, "b:e:Eho:p:Vx:", long_options, NULL)) == -1)
             break;
 
         switch (option)
@@ -429,6 +431,10 @@ static bool parse_command_line(int argc, char **argv, struct options *options)
                 options->entry_point = optarg;
                 break;
 
+            case 'E':
+                options->preprocess_only = true;
+                break;
+
             case OPTION_OUTPUT:
             case 'o':
                 options->output_filename = optarg;
@@ -669,7 +675,11 @@ int main(int argc, char **argv)
         goto done;
     }
 
-    ret = vkd3d_shader_compile(&info, &output_code, &messages);
+    if (options.preprocess_only)
+        ret = vkd3d_shader_preprocess(&info, &output_code, &messages);
+    else
+        ret = vkd3d_shader_compile(&info, &output_code, &messages);
+
     if (messages)
         fputs(messages, stderr);
     vkd3d_shader_free_messages(messages);
-- 
2.29.2

[PATCH vkd3d 3/4] vkd3d-compiler: Add support for running the HLSL preprocessor.

[Marvin]

Hi,

While running your changed tests, I think I found new failures.
Being a bot and all I'm not very good at pattern recognition, so I might be
wrong, but could you please double-check?

Full results can be found at:
https://testbot.winehq.org/JobDetails.pl?Key=82561

Your paranoid android.


=== debiant (build log) ===

Task: Patch failed to apply

=== debiant (build log) ===

Task: Patch failed to apply

[PATCH vkd3d 3/4] vkd3d-compiler: Add support for running the HLSL preprocessor.

[Henri Verbeet]

Signed-off-by: Henri Verbeet <hverbeet(a)codeweavers.com>

[PATCH vkd3d 4/4] vkd3d-compiler: Default to HLSL as a source format when preprocessing.

[Zebediah Figura]

Signed-off-by: Zebediah Figura <zfigura(a)codeweavers.com>
---
 programs/vkd3d-compiler/main.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/programs/vkd3d-compiler/main.c b/programs/vkd3d-compiler/main.c
index cec2ebb1..9aba2a26 100644
--- a/programs/vkd3d-compiler/main.c
+++ b/programs/vkd3d-compiler/main.c
@@ -397,8 +397,8 @@ static bool parse_command_line(int argc, char **argv, struct options *options)
     };
 
     memset(options, 0, sizeof(*options));
-    options->source_type = VKD3D_SHADER_SOURCE_DXBC_TPF;
-    options->target_type = VKD3D_SHADER_TARGET_SPIRV_BINARY;
+    options->source_type = VKD3D_SHADER_SOURCE_NONE;
+    options->target_type = VKD3D_SHADER_TARGET_NONE;
     options->formatting = VKD3D_SHADER_COMPILE_OPTION_FORMATTING_INDENT
             | VKD3D_SHADER_COMPILE_OPTION_FORMATTING_HEADER;
 
@@ -483,7 +483,8 @@ static bool parse_command_line(int argc, char **argv, struct options *options)
     if (options->print_target_types)
         return true;
 
-    if (!validate_target_type(options->source_type, options->target_type))
+    if (options->source_type != VKD3D_SHADER_SOURCE_NONE && options->target_type != VKD3D_SHADER_TARGET_NONE
+            && !validate_target_type(options->source_type, options->target_type))
     {
         fprintf(stderr, "Target type '%s' is invalid for source type '%s'.\n",
                 get_target_type_info(options->target_type)->name,
@@ -606,6 +607,11 @@ int main(int argc, char **argv)
         return 1;
     }
 
+    if (options.source_type == VKD3D_SHADER_SOURCE_NONE)
+        options.source_type = options.preprocess_only ? VKD3D_SHADER_SOURCE_HLSL : VKD3D_SHADER_SOURCE_DXBC_TPF;
+    if (options.target_type == VKD3D_SHADER_TARGET_NONE && !options.preprocess_only)
+        options.target_type = VKD3D_SHADER_TARGET_SPIRV_BINARY;
+
     if (options.print_version)
     {
         const char *version = vkd3d_shader_get_version(NULL, NULL);
@@ -639,8 +645,8 @@ int main(int argc, char **argv)
     if (!(output = open_output(options.output_filename, &close_output)))
         goto done;
 
-    if (!options.output_filename && get_target_type_info(options.target_type)->is_binary
-            && isatty(fileno(output)))
+    if (!options.output_filename && options.target_type != VKD3D_SHADER_TARGET_NONE
+            && get_target_type_info(options.target_type)->is_binary && isatty(fileno(output)))
     {
         fprintf(stderr, "Output is a tty and output format is binary, exiting.\n"
                 "If this is really what you intended, specify the output explicitly.\n");
-- 
2.29.2

[PATCH vkd3d 4/4] vkd3d-compiler: Default to HLSL as a source format when preprocessing.

[Marvin]

Hi,

While running your changed tests, I think I found new failures.
Being a bot and all I'm not very good at pattern recognition, so I might be
wrong, but could you please double-check?

Full results can be found at:
https://testbot.winehq.org/JobDetails.pl?Key=82562

Your paranoid android.


=== debiant (build log) ===

Task: Patch failed to apply

=== debiant (build log) ===

Task: Patch failed to apply

[PATCH vkd3d 4/4] vkd3d-compiler: Default to HLSL as a source format when preprocessing.

[Henri Verbeet]

On Thu, 26 Nov 2020 at 21:25, Zebediah Figura <zfigura(a)codeweavers.com> wrote:
> @@ -483,7 +483,8 @@ static bool parse_command_line(int argc, char **argv, struct options *options)
>      if (options->print_target_types)
>          return true;
>
> -    if (!validate_target_type(options->source_type, options->target_type))
> +    if (options->source_type != VKD3D_SHADER_SOURCE_NONE && options->target_type != VKD3D_SHADER_TARGET_NONE
> +            && !validate_target_type(options->source_type, options->target_type))
> Perhaps simply "if (!options.preprocess_only &&
> !validate_target_type(...))"? That would require resolving the default
> source and target types in parse_command_line(), but that may not be a
> bad thing.
> -    if (!options.output_filename && get_target_type_info(options.target_type)->is_binary
> -            && isatty(fileno(output)))
> +    if (!options.output_filename && options.target_type != VKD3D_SHADER_TARGET_NONE
> +            && get_target_type_info(options.target_type)->is_binary && isatty(fileno(output)))
>      {
>          fprintf(stderr, "Output is a tty and output format is binary, exiting.\n"
>                  "If this is really what you intended, specify the output explicitly.\n");
> Perhaps we should test whether the source format is binary here with
> -E. Although it may not end up being very useful in practice, I don't
> think there's an inherent reason that vkd3d_shader_preprocess()
> couldn't be used with binary formats.

[PATCH vkd3d 4/4] vkd3d-compiler: Default to HLSL as a source format when preprocessing.

[Zebediah Figura (she/her)]

On 11/27/20 7:03 AM, Henri Verbeet wrote:
> On Thu, 26 Nov 2020 at 21:25, Zebediah Figura <zfigura(a)codeweavers.com> wrote:
> > @@ -483,7 +483,8 @@ static bool parse_command_line(int argc, char **argv, struct options *options)
> >      if (options->print_target_types)
> >          return true;
> >
> > -    if (!validate_target_type(options->source_type, options->target_type))
> > +    if (options->source_type != VKD3D_SHADER_SOURCE_NONE && options->target_type != VKD3D_SHADER_TARGET_NONE
> > +            && !validate_target_type(options->source_type, options->target_type))
> > Perhaps simply "if (!options.preprocess_only &&
> > !validate_target_type(...))"? That would require resolving the default
> > source and target types in parse_command_line(), but that may not be a
> > bad thing.

That seems reasonable, sure.
> > -    if (!options.output_filename && get_target_type_info(options.target_type)->is_binary
> > -            && isatty(fileno(output)))
> > +    if (!options.output_filename && options.target_type != VKD3D_SHADER_TARGET_NONE
> > +            && get_target_type_info(options.target_type)->is_binary && isatty(fileno(output)))
> >      {
> >          fprintf(stderr, "Output is a tty and output format is binary, exiting.\n"
> >                  "If this is really what you intended, specify the output explicitly.\n");
> > Perhaps we should test whether the source format is binary here with
> > -E. Although it may not end up being very useful in practice, I don't
> > think there's an inherent reason that vkd3d_shader_preprocess()
> > couldn't be used with binary formats.

Maybe if it were something like m4, yeah, but the HLSL preprocessor is
very whitespace-sensitive. Although we could introduce a custom m4-like
preprocessor for other formats...

[Perhaps we should also add something like
vkd3d_shader_get_supported_preprocessor_types()?]

[PATCH vkd3d 4/4] vkd3d-compiler: Default to HLSL as a source format when preprocessing.

[Henri Verbeet]

On Fri, 27 Nov 2020 at 20:01, Zebediah Figura (she/her)
<zfigura(a)codeweavers.com> wrote:
> On 11/27/20 7:03 AM, Henri Verbeet wrote:
> > On Thu, 26 Nov 2020 at 21:25, Zebediah Figura <zfigura(a)codeweavers.com> wrote:
> > > -    if (!options.output_filename && get_target_type_info(options.target_type)->is_binary
> > > -            && isatty(fileno(output)))
> > > +    if (!options.output_filename && options.target_type != VKD3D_SHADER_TARGET_NONE
> > > +            && get_target_type_info(options.target_type)->is_binary && isatty(fileno(output)))
> > >      {
> > >          fprintf(stderr, "Output is a tty and output format is binary, exiting.\n"
> > >                  "If this is really what you intended, specify the output explicitly.\n");
> > > Perhaps we should test whether the source format is binary here with
> > > -E. Although it may not end up being very useful in practice, I don't
> > > think there's an inherent reason that vkd3d_shader_preprocess()
> > > couldn't be used with binary formats.
>
> Maybe if it were something like m4, yeah, but the HLSL preprocessor is
> very whitespace-sensitive. Although we could introduce a custom m4-like
> preprocessor for other formats...

Sure, we could have different preprocessors for different formats. (Or
even different preprocessors for the same format, if it came to that.)
As far as vkd3d-compiler is concerned though, none of that should
really matter; if "options.preprocess_only" is set, the format of the
output is determined by "options.source_type".
> [Perhaps we should also add something like
> vkd3d_shader_get_supported_preprocessor_types()?]
> Possibly, although at least initially we could probably get away with
> simply returning a copy of the input for source formats that do not
> have an inherent associated preprocessor.

[PATCH vkd3d 4/4] vkd3d-compiler: Default to HLSL as a source format when preprocessing.

[Zebediah Figura (she/her)]

On 11/30/20 7:05 AM, Henri Verbeet wrote:
> On Fri, 27 Nov 2020 at 20:01, Zebediah Figura (she/her)
> <zfigura(a)codeweavers.com> wrote:
> > On 11/27/20 7:03 AM, Henri Verbeet wrote:
> > > On Thu, 26 Nov 2020 at 21:25, Zebediah Figura <zfigura(a)codeweavers.com> wrote:
> > > > -    if (!options.output_filename && get_target_type_info(options.target_type)->is_binary
> > > > -            && isatty(fileno(output)))
> > > > +    if (!options.output_filename && options.target_type != VKD3D_SHADER_TARGET_NONE
> > > > +            && get_target_type_info(options.target_type)->is_binary && isatty(fileno(output)))
> > > >      {
> > > >          fprintf(stderr, "Output is a tty and output format is binary, exiting.\n"
> > > >                  "If this is really what you intended, specify the output explicitly.\n");
> > > > Perhaps we should test whether the source format is binary here with
> > > > -E. Although it may not end up being very useful in practice, I don't
> > > > think there's an inherent reason that vkd3d_shader_preprocess()
> > > > couldn't be used with binary formats.
> >
> > Maybe if it were something like m4, yeah, but the HLSL preprocessor is
> > very whitespace-sensitive. Although we could introduce a custom m4-like
> > preprocessor for other formats...
>
> Sure, we could have different preprocessors for different formats. (Or
> even different preprocessors for the same format, if it came to that.)
> As far as vkd3d-compiler is concerned though, none of that should
> really matter; if "options.preprocess_only" is set, the format of the
> output is determined by "options.source_type".

Sorry, I managed to completely misunderstand what you were saying at
first. Now this comment makes sense, and I agree, we should check the
input format here instead.
> > [Perhaps we should also add something like
> > vkd3d_shader_get_supported_preprocessor_types()?]
> > Possibly, although at least initially we could probably get away with
> > simply returning a copy of the input for source formats that do not
> > have an inherent associated preprocessor.

That's a good idea, actually; I hadn't thought of that. That does mean
that for backwards compatibility's sake there's not a lot of point in
later adding vkd3d_shader_get_supported_preprocessor_types().

[PATCH vkd3d 4/4] vkd3d-compiler: Default to HLSL as a source format when preprocessing.

[Henri Verbeet]

On Mon, 30 Nov 2020 at 20:34, Zebediah Figura (she/her)
<zfigura(a)codeweavers.com> wrote:
> On 11/30/20 7:05 AM, Henri Verbeet wrote:
> > On Fri, 27 Nov 2020 at 20:01, Zebediah Figura (she/her)
> > <zfigura(a)codeweavers.com> wrote:
> > > [Perhaps we should also add something like
> > > vkd3d_shader_get_supported_preprocessor_types()?]
> > > Possibly, although at least initially we could probably get away with
> > > simply returning a copy of the input for source formats that do not
> > > have an inherent associated preprocessor.
>
> That's a good idea, actually; I hadn't thought of that. That does mean
> that for backwards compatibility's sake there's not a lot of point in
> later adding vkd3d_shader_get_supported_preprocessor_types().

Not as long as we stick to the standard preprocessor for each source
format, no. But if we did want to add e.g. a m4 preprocessor, we could
do something like the following:

    enum vkd3d_shader_preprocessor
    {
       VKD3D_SHADER_PREPROCESSOR_NONE, /* Or
VKD3D_SHADER_PREPROCESSOR_IDENTITY, if you prefer. */
       VKD3D_SHADER_PREPROCESSOR_HLSL,
       VKD3D_SHADER_PREPROCESSOR_M4,
       ...
    };

    struct vkd3d_shader_preprocess2_info
    {
       enum vkd3d_shader_preprocessor preprocessor;
       ...
    };

    const enum vkd3d_shader_preprocessor
*vkd3d_shader_get_supported_preprocessors(
           enum vkd3d_shader_source_type source_type, unsigned int *count);

I wouldn't necessarily expect a lot of demand for that feature though.

[Marvin]

Hi,

While running your changed tests, I think I found new failures.
Being a bot and all I'm not very good at pattern recognition, so I might be
wrong, but could you please double-check?

Full results can be found at:
https://testbot.winehq.org/JobDetails.pl?Key=82559

Your paranoid android.


=== debiant (build log) ===

Task: Patch failed to apply

=== debiant (build log) ===

Task: Patch failed to apply

[Henri Verbeet]

Signed-off-by: Henri Verbeet <hverbeet(a)codeweavers.com>