vkd3d: Thoughts on how to include shaders in the project?
Hello, I'm currently looking into getting Deus Ex: Mankind Divided to run on vkd3d. Among other issues, it uses ClearUnorderedAccessView for typed buffer views, which is currently not supported and will require the use of a compute shader. There are also a few other areas where shader-based implementations might become necessary in the future, e.g. for copies between depth and color images, ResolveSubresource with typeless images, etc. What would be the preferred way to integrate shaders for this purpose in vkd3d? One option would be to compile GLSL shaders at build-time to a C header containing the SPIR-V code, which could then be included in the vkd3d source files directly. This would introduce glslangValidator as a new build-time dependency, is that acceptable? Also, how would this be implemented in the build system? I don't have much experience with autoconf, so some help in that regard would be highly appreciated. I'd like to avoid including pre-compiled SPIR-V binaries in the source tree directly or installing them as separate files if possible, since doing so would complicate the workflow significantly. Regards, - Philip
Hi Philip, On Mon, 4 Nov 2019 at 19:16, Philip Rebohle <philip.rebohle(a)tu-dortmund.de> wrote:
I'm currently looking into getting Deus Ex: Mankind Divided to run on vkd3d. Among other issues, it uses ClearUnorderedAccessView for typed buffer views, which is currently not supported and will require the use of a compute shader.
There are also a few other areas where shader-based implementations might become necessary in the future, e.g. for copies between depth and color images, ResolveSubresource with typeless images, etc.
What would be the preferred way to integrate shaders for this purpose in vkd3d?
I think the preferred way would be to include them as HLSL source, and use vkd3d-shader to compile them to SPIR-V. The tricky part there is that the HLSL compiler is currently still a work in progress. It's possible to use Microsoft's fxc as a replacement—we've used that for vkmodelviewer in the past—but it seems undesirable to have fxc as a dependency for vkd3d.
One option would be to compile GLSL shaders at build-time to a C header containing the SPIR-V code, which could then be included in the vkd3d source files directly.
This would introduce glslangValidator as a new build-time dependency, is that acceptable? Also, how would this be implemented in the build system? I don't have much experience with autoconf, so some help in that regard would be highly appreciated.
In principle, sure. We used to have a dependency on glslangValidator in the past for the demos, but that was removed in commit aa5d48eec4987c4c1cb55571b536cd485c25740b once vkd3d-shader became capable of translating the TPF/DXBC shaders.
I'd like to avoid including pre-compiled SPIR-V binaries in the source tree directly or installing them as separate files if possible, since doing so would complicate the workflow significantly.
What we typically do (e.g. in the tests) is to include HLSL source in a comment, and then setup editor commands to generate the equivalent blob inline. It's not too bad once you have things set up, but it is a bit of a pain, yes. Henri
Am 04.11.19 um 17:12 schrieb Henri Verbeet:
Hi Philip,
On Mon, 4 Nov 2019 at 19:16, Philip Rebohle <philip.rebohle(a)tu-dortmund.de> wrote:
I'm currently looking into getting Deus Ex: Mankind Divided to run on vkd3d. Among other issues, it uses ClearUnorderedAccessView for typed buffer views, which is currently not supported and will require the use of a compute shader.
There are also a few other areas where shader-based implementations might become necessary in the future, e.g. for copies between depth and color images, ResolveSubresource with typeless images, etc.
What would be the preferred way to integrate shaders for this purpose in vkd3d?
I think the preferred way would be to include them as HLSL source, and use vkd3d-shader to compile them to SPIR-V. The tricky part there is that the HLSL compiler is currently still a work in progress. It's possible to use Microsoft's fxc as a replacement—we've used that for vkmodelviewer in the past—but it seems undesirable to have fxc as a dependency for vkd3d.
Is there any particular reason you're suggesting HLSL? Given that these shaders would sort out some of the gritty implementation details and we want full control over how exactly resources are accessed, I'd think having a more direct path without going through DXBC and especially the peculiarities of the D3D12 binding model would be preferable.
One option would be to compile GLSL shaders at build-time to a C header containing the SPIR-V code, which could then be included in the vkd3d source files directly.
This would introduce glslangValidator as a new build-time dependency, is that acceptable? Also, how would this be implemented in the build system? I don't have much experience with autoconf, so some help in that regard would be highly appreciated.
In principle, sure. We used to have a dependency on glslangValidator in the past for the demos, but that was removed in commit aa5d48eec4987c4c1cb55571b536cd485c25740b once vkd3d-shader became capable of translating the TPF/DXBC shaders.
I'd like to avoid including pre-compiled SPIR-V binaries in the source tree directly or installing them as separate files if possible, since doing so would complicate the workflow significantly.
What we typically do (e.g. in the tests) is to include HLSL source in a comment, and then setup editor commands to generate the equivalent blob inline. It's not too bad once you have things set up, but it is a bit of a pain, yes.
Right, I might just do this for now to get things started. - Philip
Henri
On Mon, 4 Nov 2019 at 21:08, Philip Rebohle <philip.rebohle(a)tu-dortmund.de> wrote:
Am 04.11.19 um 17:12 schrieb Henri Verbeet:
I think the preferred way would be to include them as HLSL source, and use vkd3d-shader to compile them to SPIR-V. The tricky part there is that the HLSL compiler is currently still a work in progress. It's possible to use Microsoft's fxc as a replacement—we've used that for vkmodelviewer in the past—but it seems undesirable to have fxc as a dependency for vkd3d.
Is there any particular reason you're suggesting HLSL? Given that these shaders would sort out some of the gritty implementation details and we want full control over how exactly resources are accessed, I'd think having a more direct path without going through DXBC and especially the peculiarities of the D3D12 binding model would be preferable.
I have no particular fondness for HLSL, but it would mean one less external dependency, and peculiarities aside, HLSL should broadly have equivalent capabilities to GLSL. That's in the abstract though, I haven't seen the specifics of the shaders in question, of course.
On 11/4/19 4:46 PM, Philip Rebohle wrote:
Hello,
I'm currently looking into getting Deus Ex: Mankind Divided to run on vkd3d. Among other issues, it uses ClearUnorderedAccessView for typed buffer views, which is currently not supported and will require the use of a compute shader.
There are also a few other areas where shader-based implementations might become necessary in the future, e.g. for copies between depth and color images, ResolveSubresource with typeless images, etc.
What would be the preferred way to integrate shaders for this purpose in vkd3d?
One option would be to compile GLSL shaders at build-time to a C header containing the SPIR-V code, which could then be included in the vkd3d source files directly.
This would introduce glslangValidator as a new build-time dependency, is that acceptable? Also, how would this be implemented in the build system? I don't have much experience with autoconf, so some help in that regard would be highly appreciated.
I can suggest using shaderc, as glslc has a built-in option to emit a C array, using -mfmt=c, which can be #included into a C file. Doing the same in glslangValidator might be more work overall. Cheers, Hans-Kristian
I'd like to avoid including pre-compiled SPIR-V binaries in the source tree directly or installing them as separate files if possible, since doing so would complicate the workflow significantly.
Regards,
- Philip
Am 04.11.19 um 17:23 schrieb Hans-Kristian Arntzen:
On 11/4/19 4:46 PM, Philip Rebohle wrote:
Hello,
I'm currently looking into getting Deus Ex: Mankind Divided to run on vkd3d. Among other issues, it uses ClearUnorderedAccessView for typed buffer views, which is currently not supported and will require the use of a compute shader.
There are also a few other areas where shader-based implementations might become necessary in the future, e.g. for copies between depth and color images, ResolveSubresource with typeless images, etc.
What would be the preferred way to integrate shaders for this purpose in vkd3d?
One option would be to compile GLSL shaders at build-time to a C header containing the SPIR-V code, which could then be included in the vkd3d source files directly.
This would introduce glslangValidator as a new build-time dependency, is that acceptable? Also, how would this be implemented in the build system? I don't have much experience with autoconf, so some help in that regard would be highly appreciated.
I can suggest using shaderc, as glslc has a built-in option to emit a C array, using -mfmt=c, which can be #included into a C file. Doing the same in glslangValidator might be more work overall.
glslangValidator also provides --variable-name to generate a C array, which I'm using in DXVK without too much of a hassle. Does shaderc do anything differently? - Philip
Cheers, Hans-Kristian
I'd like to avoid including pre-compiled SPIR-V binaries in the source tree directly or installing them as separate files if possible, since doing so would complicate the workflow significantly.
Regards,
- Philip
On 11/4/19 5:27 PM, Philip Rebohle wrote:
Am 04.11.19 um 17:23 schrieb Hans-Kristian Arntzen:
On 11/4/19 4:46 PM, Philip Rebohle wrote:
Hello,
I'm currently looking into getting Deus Ex: Mankind Divided to run on vkd3d. Among other issues, it uses ClearUnorderedAccessView for typed buffer views, which is currently not supported and will require the use of a compute shader.
There are also a few other areas where shader-based implementations might become necessary in the future, e.g. for copies between depth and color images, ResolveSubresource with typeless images, etc.
What would be the preferred way to integrate shaders for this purpose in vkd3d?
One option would be to compile GLSL shaders at build-time to a C header containing the SPIR-V code, which could then be included in the vkd3d source files directly.
This would introduce glslangValidator as a new build-time dependency, is that acceptable? Also, how would this be implemented in the build system? I don't have much experience with autoconf, so some help in that regard would be highly appreciated.
I can suggest using shaderc, as glslc has a built-in option to emit a C array, using -mfmt=c, which can be #included into a C file. Doing the same in glslangValidator might be more work overall.
glslangValidator also provides --variable-name to generate a C array, which I'm using in DXVK without too much of a hassle. Does shaderc do anything differently?
If glslang also has support for this now, then there isn't much of a point in using shaderc. Cheers, Hans-Kristian
- Philip
Cheers, Hans-Kristian
I'd like to avoid including pre-compiled SPIR-V binaries in the source tree directly or installing them as separate files if possible, since doing so would complicate the workflow significantly.
Regards,
- Philip
participants (3)
-
Hans-Kristian Arntzen -
Henri Verbeet -
Philip Rebohle