The discussion about that has mostly been along the lines of having separate sm1 and sm4 IR, although it's been vague and I think it's reasonable that we could use a common struct on the same level as vkd3d_shader_instruction instead. I don't think there's any reason that we want anything higher-level than that (with the possible caveat that maybe we want to use pointers rather than register numbers?), particularly because most of the impetus for introducing this new IR is that the current HLSL IR is *too* high-level (e.g. around things like variable loads and stores, and not being able to express the more CISC aspects of the smX assembly, like source modifiers.)
I doesn't need to be a much higher level as far as I'm concerned, but a few issues I can think of with the current scheme:
- Declarations being part of the same instruction stream as the rest of the shader can be a bit awkward; particular for shader model 1-3 which doesn't necessarily have them. I suspect you're either going to run into this with your d3dbc->spirv efforts, or already have. In wined3d that's somewhat addressed by constructing the wined3d_shader_reg_maps structure and then ignoring declaration instructions in the GLSL backend.
In terms of translating sm1->spirv (and in general translating out of vkd3d_shader_instruction), sort of, although for the most part spirv.c should be capable of lazily initializing varyings, and most of my difficulty with sm1 thus far has been rearranging it so that it will. (And also so that it doesn't demand an input signature.)
In terms of translating hlsl->smX (and in general translating from something more high-level into vkd3d_shader_instruction), I don't think declarations are awkward at all?
I can definitely see this being an improvement—it'd make the spirv code for handling declarations less complicated—but as below, it'd also mean doing more passes, and probably more allocations as well.
- There are some differences in behaviour between the same instructions in various shader models. (E.g., mov/mova, exp/log/rcp/rsq, sincos, various tex* instructions.) We currently expect the backends to be aware of those, but it may be nicer to smooth over that in the frontends.
Agreed, we could definitely be doing more than we are right now.
- Somewhat similar to declaration instructions, it probably makes sense to make hull shader phases available as separate blocks of instructions, instead of a single instruction stream.
I can't much comment on this as I have thus far avoided touching or understanding tessellation. God only knows why it's so complicated...
- We typically end up parsing the same shader bytecode multiple times anyway. scan_with_parser() does a pass, vkd3d_shader_trace() may do a pass, and then the actual translation does a pass. (Though note that technically that's not a limitation of the interface as such; it would be entirely possible for a particular frontend to parse the shader only once and return existing instructions from parser_read_instruction().)
Hmm, right, I forgot we do two passes already. There's still basically no allocation from the smX frontends, though, and I think it'd be hard to improve on that...
Note that I'm quite explicitly not suggesting to throw out vkd3d_shader_instruction and replacing it with something new; the suggestion is that the vkd3d_shader_instruction interface could likely be made to work for both HLSL and DXIL with a reasonable number of adjustments. If HLSL could use it as-is, that's all the better.
Potentially. The main reason I haven't thus far is that it's extra work (if not per se a *lot* of extra work) for no clear benefit, versus our more ad-hoc struct smX_instruction infrastructure. If there were other frontends that wanted to generate sm1/sm4, or a reason for hlsl to feed directly to glsl or spirv instead of going through sm4 first, that'd more easily tip the scales.
If we do want to adopt it in more places, I support throwing out the vkd3d_shader_instruction naming and replacing it with something new :D
Maybe there's an argument to have one or more unified IRs across all of vkd3d-shader? It would be nice in some respects, but I imagine that compilation speed would be a concern. I gather that one reason that the dxbc->glsl/spirv path is arranged the way it is, is that we only want to do one pass over the dxbc, and want to avoid allocating memory as much as we can.
Well, things started out as directly translating shader model 1 bytecode to ARB_vertex_program/ARB_fragment_program instructions. (Compare e.g. IWineD3DVertexShaderImpl_GenerateProgramArbHW() from dlls/wined3d/vertexshader.c in wine-0.9.) Certain abstractions were introduced as needed; the introduction of the GLSL backend was an important event, as was the introduction of shader model 4 support. We've probably reached a similar point again, although this time it seems both DXIL and HLSL are getting there at roughly the same time.
I guess my point is, if we have a unified IR and it's basically vkd3d_shader_instruction, that's probably fine (and it may honestly be possible to do that, the way things are). But if we want to change vkd3d_shader_instruction, that might mean making the sm4->spirv path slower, which doesn't seem desirable.