I've pushed an update of the shader cache patches. Sorry Gio, it is a mix of rebase and new cache work.
It has a big amount of patches now that can be roughly grouped as follows:
1-9: Define and implement the shader cache API itself 10-20: Use the cache to store vkd3d pipelines 21-23: Some refinements we may or may not want (esp 22 is fairly crossover centric) 24-28: Tests 29: Benchmark / debug hacks. We don't want to merge them as such
I think I got out the obvious TODOs, lack-of-error-handling FIXMEs etc. There are still some tasks left before this is ready to be merged (see below), but I don't think they are in the way of another round of review.
Major changes since the last version:
1) I've removed the hash collision return value and compare the entire key in the tree compare callback. Extra care will need to be taken once we add memory evicition, I have added a comment to that effect. The reason is that wined3d can replace some of its own shader rbtrees with an in-memory cache as long as the cache doesn't throw away data randomly (which a hash collision de facto does).
2) Plenty of work for a non-change: No sqlite. I looked into it, and we can certainly use it to write our files, but it is not suitable for replacing our own memory data structures. The performance of sqlite for a simple key-value lookup is a lot worse than the rbtree. Furthermore it can't stop being a transactional database system that writes to disk at the earliest oportunity. This makes cache writes for on-disk caches slow, although this is a property that native d3dscache shares.
If we write the entire cache in one transaction (i.e., keep our own rbtree and just flush it out in shader_cache_write in one go) it is fast. The sqlite way of integrating it is to put the 8 MB sqlite3.c 'amalgamation' into our source tree, but with our default compiler settings it spews out plenty of warnings about discarded consts and the like. The Wine way would be to put said sqlite3.c into wine/dlls/winsqlite3.dll and link to it. I don't want to add this delay right now, but I can certainly change my mind if there's consensus to do either of those two things.
Another concern with sqlite is the address space use. Its documentation specifies a number of knobs to control its memory consumption, but I didn't look if they actually do what we want.
3) I added a flag to disallow replacing an existing value.
4) I nicked VKD3D_MUTEX_INITIALIZER from !384.
5) Vulkan-style extended structures
---
Some things I didn't do (yet). I don't think they influence review much:
1) Relocate everything to vkd3d-shader. Or figure out the d3dscache.dll API and implement that. It doesn't look complicated. I think we want to sort this out before un-drafting this MR.
2) Size accounting. This is where vkd3d_shader_cache_{add/remove}_item come in in the future.
3) Cache eviction / partial reads. Not a major issue for me at this point, but we want this sorted before using it in wined3d (due to 32 bit concerns).
4) Still using mvk's hashing function I don't think we need to change away from 64 bit hashes now that the cache always compares the full key.
5) A separate flag for memory only caches vs using a disk size of 0. I don't care, Matteo and Gio fight this out outside of the saloon at high noon :-)
6) Add a flag for a private only cache / make sharing opt-in as suggested by Gio: I have no strong opinion either way. I can't think of a case where we want in-process cache sharing ourselves, otoh it is yet another behavior modifying flag in the API.