I agree that `assert()` is not an appropriate way to deal with unexpected or untrusted input. My point is that that `assert()` should never trigger, even on unexpected or untrusted input, because I consider the frontend's job to deal with such input.
Well, yes, but that's what makes it unexpected, as opposed to just invalid.
In other words, my expectation from the frontend is that it's free to reject its input (and in that case no IR is generated nor validated), but if it accepts the input then it takes the responsibility to generate correct and valid IR; so the IR read by the validator is already considered trusted, and the `assert()` call is there just to, well, assert that. If we don't `assert()` we're likely in UB domain anyway, because the backend will assume that the IR is correct and count on it. If the IR is not valid vkd3d could crash anyway, and by that point I believe it's better to crash in a repeatable way with a clear error message rather then ending up tripping whatever UB will bring us any time later.
That's not that choice here though. I have no issue with aborting compilation when validation fails; the issue is with using assert() to do that, because it may either not trigger (i.e., in NDEBUG builds) or potentially escalate the consequences by calling abort() in an inappropriate context.
However, I don't mean to insist too much on this detail. I can remove the assertion and just rely on logging;
See above; I think we should (possibly optionally) fail compilation and return an error.
I can also gate the check with `TRACE_ON()`, which hopefully addresses your performances concern. I should mention that at some point I contemplated making this a `FIXME` rather than a `TRACE`, so that it is more conspicuous on logs posted by users, but that makes sense only if we accept the validator to be always run.
TRACE_ON is a bit inconvenient for one of the usages I had in mind. In particular, assume I have a collection of shaders collected using VKD3D_SHADER_DUMP_PATH, and I'd like to validate that these produce valid output, for example before approving certain MRs or making a release. Ideally I would be able to run vkd3d-compiler on each of these shaders, check its exit code and log its output, and get useful information out of that. This is an issue with the existing vkd3d_spirv_validate() as well, of course; in that particular case the existence of spirv-val helps.
Using an environment variable would work; so would introducing a compilation option.