In any case, while it's perhaps not quite true at the moment, parsing and serialising the different bytecode formats should be closely related and use the same constants, structures, and so on. There's currently some HLSL-specific code intermixed that would more properly belong in hlsl.c/hlsl_codegen.c, but we can deal with that once we get to it.
...Possibly. I would have appreciated a bit more discussion on this first. I had been vaguely thinking of moving more code *to* the backend files, so now I need to stop and re-plan.
And while I think we'd always been talking about using the same structures in hlsl_smX, I kind of thought that we would perform that conversion before moving things around. That would have let things be done more gradually, and avoided some unpleasant rebases.
Anyway, generally this ties into the question I brought up in [1]. I brought up a relatively complete summary of our options with the IR there, but it doesn't seem to have garnered any discussion. At best we seem to have committed to a "maximally CISC" instruction set with Conor's normalization patches, so that rules out option (2).
Well, yeah, the impression I came away with from that conversation was that there were no particularly strong preferences, and not much of a general consensus either; few people seem to care. I don't think we've ruled out any of the options presented there at this point. As noted there, (1) is essentially the current state, but we could certainly move away from that towards one of the other options.
Yeah, that makes sense. It's just that there are several vkd3d developers rather experienced in design—and also often opinionated—and I was hoping to get at least some thoughts on the matter :-/
And ultimately I think the distinction between the options is a bit artificial to begin with; it ends up largely being a matter of doing particular transformation passes on either the frontend IR, the intermediate IR, or the backend IR. That's not a decision we need to make upfront for every conceivable transformation, and we can change our minds if a particular choice doesn't quite work out. In the case of Conor's tessellation shader work, the consideration is simply that these transformations are more practical to do on the vkd3d_shader_instruction level than on the SPIR-V level, and while doing them on the TPF level would also be plausible, that would require doing a bunch of disassembler work first.
I think it's less artificial than it looks, largely because we only have the one target (SPIRV) that would involve lowering transformations. Which is to say, there are a lot of lowering transformations that we could be doing, rather than (as I suspect we will initially) independently lowering things in GLSL and SPIRV. Similarly there are a lot of raising transformations we could be doing in HLSL -> smX that we currently aren't.
I think there's also a lot to be said for being consistent about where we do a given transformation, but maybe I'm cargo-culting consistency a bit too hard.
The effect of all this is that while it's not hard to adjust, it could eventually involve rewriting a *lot* of transformations, and so it seems like the kind of thing to think about now before we commit to one approach.