This is probably the most ugly and controversial bit of API. I don't really know
if this is the right approach to solving this.
sm4 registers match by register index, such that shaders can mostly be compiled
in isolation. sm1 does not—registers may be specified in any order in the vertex
and pixel shaders, and will be matched by usage and usage index.
By itself this is not much of a problem. Where it gets hairy is that we want to
do some degree of caching, as well as pre-compilation, and avoid recompiling
either shader every time it's matched with a new one.
Wine currently deals with this problem, for GLSL, by generating a "main" GLSL
shader for the vertex shader, and then an extra function setup_vs_output(), and
linking the two together every time a new pixel shader is used. This could in
theory be used for SPIR-V, but it requires the use of extra, probably external,
code to link SPIR-V shaders together, which I do not particularly anticipate
being well-received.
(I'm not sure how Wine deals with this problem in the ARB backend. It seems to
take the pixel shader signature into account when generating the vertex shader—
cf. init_output_registers()—but it doesn't take it into account when looking up
a vertex shader variant? I didn't look too closely at the code, so maybe I'm
missing something.)
--
The vkd3d parts of this patch are quite straightforward, and looking at them, I
think the design is quite intuitive in isolation. There may be some room for
internal refactoring (in particular with an eye to not so much overloading
the "register_index" field of struct shader_signature_element) but I'm
relatively happy with the way it turned out. In isolation, that is.
The Wine part is worse. I've uploaded branches for vkd3d and Wine that use this
API, and correctly handle shaders with some nontrivial reordering:
https://gitlab.winehq.org/zfigura/vkd3d/-/commits/himavant5https://gitlab.winehq.org/zfigura/wine/-/commits/himavant_cb
The test can be run, as before, like so:
make tests/shader_runner.exe && WINE_D3D_CONFIG=renderer=vulkan wine tests/shader_runner.exe ../vkd3d/tests/hlsl/sm1-interstage-interface.shader_test
The interesting Wine parts are concentrated in a single patch, 5cfb9d930f11e.
The patch takes a few shortcuts, partly because I wanted not to block the vkd3d
API design questions, but also because while writing it I came up with a couple
of problems that I wasn't sure how to fix. There are two main problems I see:
(1) This patch has the user pass the signature from the pixel shader when
compiling the vertex shader, and looks up register indices already
arbitrarily allocated by the pixel shader. This is problematic when trying
to use this signature as a cache key, by virtue of it not being clear (or
even defined) which fields are key elements and which aren't. It's also not
particularly kind to lookup, on account of not being directly comparable
with memcmp(). There are a few options I see:
(a) Provide an internal function to compare key elements. This feels... odd,
like a very special-purpose function, but perhaps workable.
(b) Just make the user deal with it, and assert that all fields are key
elements.
(c) Use some alternative, perhaps shortened structure as a field of
vkd3d_shader_next_stage_info. This has the disadvantage that it is not
as simple for a hypothetical user to retrieve from the pixel shader, but
we would presumably provide a function to generate one from a shader
signature. This would probably also be kinder to cache lookup if it's
shorter.
(d) Make caching vkd3d's responsibility, to some degree. This seems
daunting, but the more we optimize, the more difficult it may be to
design API that allows for nice caching.
(2) Assuming we use signatures, there is a memory management problem that
5cfb9d930f11e spells out. This is probably a matter of "just fix it", but
I suppose another option is to take the GLSL or ARB architecture.
--
April is the cruellest month, breeding
Lilacs out of the dead land, mixing
Memory and desire, stirring
Dull roots with spring rain.
--
v2: vkd3d-shader: Implement a function to build a varying map between sm1 stages.
vkd3d-shader: Implement remapping shader output registers to match the next shader's semantics.
https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/280
--
v11: vkd3d-shader/tpf: Add support for writing 'resinfo' instruction.
vkd3d-shader/tpf: Add support for writing 'sampleinfo' instruction.
vkd3d-shader/hlsl: Parse GetDimensions() method.
tests: Add some tests for GetDimensions().
https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/218
DirectX's FFP normalizes the zero vector to the zero vector, but GLSL
normalizes them to all NaN.
This patch creates `ffp_normalize`, which normalizes vectors but
has DirectX's behavior on 0 vectors. Further patches in this set switch
other calls from GLSL's `normalize` to the 0-safe version.
This fixes [36564](https://bugs.winehq.org/show_bug.cgi?id=36564).
--
v2: wined3d: Use ffp_normalize in shader_glsl_ffp_vertex_lighting_footer.
wined3d: Use ffp_normalize in shader_glsl_ffp_vertex_lighting.
wined3d: Implement a zero-safe normalize function for FFP.
https://gitlab.winehq.org/wine/wine/-/merge_requests/3419
When the sample size is too small, we were releasing it from the memory,
but kept the next_sample pointer set. Later, when the transform removes
the sample from the allocator, its refcount was decremented one more
time, effectively leaking the sample.
--
https://gitlab.winehq.org/wine/wine/-/merge_requests/3427
On Thu Jul 27 20:38:34 2023 +0000, Zebediah Figura wrote:
> > Beyond the test included in the patches, I did roughly the following:
> ptr = VirtualAlloc(NULL 4096 \* 2, PAGE_NOACCESS); VirtualProtect(ptr,
> 4096, PAGE_READWRITE); params = ptr + 4096 - sizeof(\*params). And used
> that pointer. So far it doesn't access it beyond the end of accessible
> memory (that succeeds).
> Ah yes, that definitely sounds convincing to me :-)
> > It might happen that with some other values for unknown parameter it
> is using more data from the structure, but IMO exploring more details
> about those parameters, in the absence of any known app which uses those
> functions, looks like a waste of time to me. It is relatively easy here
> with nsi and was already explored for other functions so I didn't want
> to break this tradition in nsi, but in general it seems impractical to
> me to spend huge amount of time reversing the exact parameters of the
> undocumented functions which nothing is using directly. Not feasible for
> implementing larger APIs and the benefit is not obvious. And once
> something is using directly in a way we don't support we can rather
> easily look how and test with those parameters.
> Sure. I don't think we necessarily need to figure out the unknown
> parameter, or spend too much effort on reverse-engineering undocumented
> things in general, or closely matching undocumented APIs.
> I only get nervous about the prospect of implementing undocumented APIs
> that are incorrect in ways that would be subtle or hard to debug—wrong
> number of parameters, wrong struct sizes, and so on. To a degree, the
> fact that an application even uses such APIs means that it's a first
> place to look when debugging, but that's not exactly the kind of thing
> that's trivial to notice.
> > I could possibly add a fixme for nonzero unk parameter?
> That sounds like a unilaterally good idea to me, at least.
Now when this patchset is split in two this first part has only a stub FIXME there anyway; I will update the second part with the specific fixme for unknown parameter.
--
https://gitlab.winehq.org/wine/wine/-/merge_requests/3423#note_40573
> Beyond the test included in the patches, I did roughly the following: ptr = VirtualAlloc(NULL 4096 \* 2, PAGE_NOACCESS); VirtualProtect(ptr, 4096, PAGE_READWRITE); params = ptr + 4096 - sizeof(\*params). And used that pointer. So far it doesn't access it beyond the end of accessible memory (that succeeds).
Ah yes, that definitely sounds convincing to me :-)
> It might happen that with some other values for unknown parameter it is using more data from the structure, but IMO exploring more details about those parameters, in the absence of any known app which uses those functions, looks like a waste of time to me. It is relatively easy here with nsi and was already explored for other functions so I didn't want to break this tradition in nsi, but in general it seems impractical to me to spend huge amount of time reversing the exact parameters of the undocumented functions which nothing is using directly. Not feasible for implementing larger APIs and the benefit is not obvious. And once something is using directly in a way we don't support we can rather easily look how and test with those parameters.
Sure. I don't think we necessarily need to figure out the unknown parameter, or spend too much effort on reverse-engineering undocumented things in general, or closely matching undocumented APIs.
I only get nervous about the prospect of implementing undocumented APIs that are incorrect in ways that would be subtle or hard to debug—wrong number of parameters, wrong struct sizes, and so on. To a degree, the fact that an application even uses such APIs means that it's a first place to look when debugging, but that's not exactly the kind of thing that's trivial to notice.
> I could possibly add a fixme for nonzero unk parameter?
That sounds like a unilaterally good idea to me, at least.
--
https://gitlab.winehq.org/wine/wine/-/merge_requests/3423#note_40571