On Tue, Jun 22, 2021 at 9:32 PM Zebediah Figura (she/her) zfigura@codeweavers.com wrote:
On 6/22/21 1:07 PM, Matteo Bruni wrote:
I think we're past the point where we need to figure out what we want to do with the d3dx* DLLs, particularly regarding how to avoid duplication of common functionality between d3dx9, d3dx10 and d3dx11. This is something that's been on my mind for a very long time but it's generally been a topic where people either care strongly or not at all, so I kept pushing back the discussion over and over again. Now I'm finally asking the former category to comment. The latter can stop reading now :P TL;DR: jump to the second-to-last paragraph.
Let's start with a quick summary of what those DLLs are. They are basically a loose collection of utility APIs that might often come in handy when writing 3d applications. To give a broad idea, these APIs can be roughly categorized into math, mesh, shader, texture functions. A d3dxn is somewhat related to the corresponding d3dn version: for example d3dx9's D3DXCreateTextureFromFileA() creates a IDirect3DTexture9 object and fills it with data from an on-disk image file. Many other APIs (e.g. the math functions) are abstract enough to be independent from any d3d version. A particularly important detail with these DLLs is what MS decided to do with versioning: each new release of the DirectX SDK (when those were still a thing) usually had a new version of the d3dx DLLs as entirely separate files. There might have been additions or changes compared to a previous version but applications wouldn't see any of that unless they intentionally switched to linking with the new DLL. For example, the last released DirectX SDK (SDK version 43) contains d3dx9_43.dll, d3dx10_43.dll, d3dx11_43.dll, in addition to ALL the DLLs from the previous SDKs (which for d3dx9 means going back up to d3dx9_24.dll). There are interesting side effects of this choice. It allowed MS to make incompatible changes (e.g. see the ID3DXEffect vtbl). It also means that applications can end up depending on quirks specific to a particular d3dxn_xx.dll version. It's also not uncommon to see applications that load and possibly use multiple d3dx DLLs inside the same process. In general, it's lots of fun...
Now let's go through the current state of those DLLs in Wine. The d3dx9 DLLs are in a pretty good state as far as API coverage goes (with a few notable exceptions, e.g. shader compiler). The implementation is in d3dx9_36/ and all the other _xx variants use the PARENTSRC mechanism to build entirely separate DLLs, mimicking native. Originally we forwarded everything to d3dx9_36 but that broke some applications, particularly from the demo scene. d3dx10 still forwards most of the math functions to d3dx9_36. The rest is largely unimplemented. d3dx11 is all but unimplemented.
Fortunately, MS dropped most of the APIs along the way, especially when moving from d3dx10 to d3dx11. Unfortunately, that doesn't help a ton. For example, there is enough stuff in the texture API alone that could (and should) be shared and the API itself is different enough that it can't be done by simply forwarding some public function from d3dxn to d3dx(n-1) or such. Having >= 3 similar but separate implementations of image filtering over various d3dx DLLs that all do the same thing seems not that attractive. And then let's not forget the other "axis" i.e. the _xx suffixed versioning which can make things extra convoluted.
If you're still here, congratulations! Now for the proposal i.e. how to move forward. There are a few options I can think of: 1 - Allow to have multiple paths in PARENTSRC, rework the d3dx9 code to make it possible to reuse helpers in d3dx10 and d3dx11 2 - Introduce a new PE DLL where the shared code would reside, keep the PARENTSRC mechanism to handle the _xx versioning 3 - Introduce a new unix lib for the shared code, keep the PARENTSRC mechanism for the _xx versioning part 4 - Just duplicate everything, durr
Reasons to favor one option over the rest? Other ideas I have missed? Personally I prefer option 2. The only downside that I see is that it would introduce a Wine-specific DLL, which is something that we normally want to avoid. There are many advantages though: compared to 1 (and the status quo) we'd reduce Wine compilation time and the overall size of the d3dx DLLs; we'd also avoid loading extra d3dx* DLLs as required by the forwards. In respect to option 3, we wouldn't introduce unnecessary (sys)call overhead, which we really don't want to have for stuff like D3DXVec4Transform(). Looking forward to any type of comment.
Personally I'm a strong proponent of option 2. I know Wine is very reticent to add custom DLLs where not necessary, and I'm not sure I understand why—maybe there are historic reasons for this?
And if there are reasons we can't use a shared library, we could at least use a static library.
Yeah, a static library would be yet another option, although I'd really prefer if we could share code (on disk AND at runtime, it's really common to find games that have 2 or more d3dx9 DLLs mapped in their process).
For the record, d3dx* is huge as-is. Each DLL is already large (for 64-bit, 3 MB), and the sum total is the largest single chunk of the system32 directory (62 MB out of 437 MB. For comparison, all of the msvcrt modules together are 31 MB, and so is all of msvcp.) And then compilation is very slow...
Yes, absolutely. It's also very wasteful, we're recompiling basically the same code 20 times. It just doesn't feel right...
Incidentally, how much of d3dx can/should live in vkd3d-shader? Off the top of my head:
- D3DX{Assemble,Compile,Preprocess}Shader() obviously, but that's
already forwarding to d3dcompiler anyway. Also D3DXDisassembleShader() and ID3DXEffectCompiler::Compile{Effect,Shader}().
- The ID3DXConstantTable implementation probably shouldn't, but maybe
parse_ctab_constant_type() should? But given the extra translation it may not in fact save us any work.
Fragment linking? I don't know how fragment linking works.
Texture shaders? I don't know how texture shaders work.
d3dx_parse_effect()?
Anything else? I can't immediately think of anything else related to
shaders.
Also this. I didn't mention it explicitly but one could think of moving the bulk of d3dx over to vkd3d[-shader] (an option 3.1, if you will). I agree that, aside from some more shader stuff, most of it doesn't seem to be a good fit for vkd3d IMHO.
WRT other d3dx pieces for vkd3d-shader, at this point I'm unsure but yes, maybe parts of the constant table parsing / handling code could reside there. More than that seems questionable.
OT: speaking of those weird / rare shader-related APIs, I don't know a lot either. In fragment linking, AFAIU, you first pass a HLSL shader that contains a number of fragments (what are those? functions? raw blocks of code? Your guess is as good as mine) to D3DXGatherFragments() and obtain some kind of rich bytecode with a bunch of compiled fragments and some kind of specification of the interface of each fragment. Then, at some point down the line, you call D3DXCreateFragmentLinker() and ID3DXFragmentLinker::AddFragments() on the returned object, passing in the fragments you previously compiled. Finally, whenever you're ready, you call ID3DXFragmentLinker::LinkShader() specifying which fragments you want to use and you get the final, combined shader bytecode. This clearly needs some kind of support from the HLSL compiler, but I suspect the linking itself to be somewhat self contained.
I know a bit more about texture shaders. They are basically a way to initialize the contents of a texture by means of a shader. Specifically the HLSL source is compiled into effect framework's "preshader" bytecode and that's executed (presumably by the same "preshader VM" as the effect evaluators) for each pixel of the texture. So it seems reasonable that the HLSL compiler would need to be able to generate "preshader" bytecode (and probably disassemble it too) but otherwise the rest should be strictly d3dx stuff.
Not that I expect any of these to happen in the near future but, since we're doing some large-scale planning, might as well keep everything in consideration...
ἔρρωσθε, Zebediah