RFC: VK_EXTERNAL_MEMORY_WIN32 extension patch

[Derek Lesho]

Hello, before I break this patch up and submit a patchset, I'd like to
inquire about a few things in this patch I have made.

The biggest question I have is how we want to process allocate_info without
modifying what is passed in.

In order to copy the whole pNext chain or the unknown bits, we'd need a way
to get the struct length given the sType.  To do this we'd have to generate
something in make_vulkan, but I'm not sure of the specifics (mostly where
the helper would go).

Alternatively, we could try restoring the chain after it is finished,
however this may not be thread safe, if the struct is being read in another
thread.

Additionally, I'd like to know how to better organize the functions for
creating/operating on gpu-resource objects.  Right now they're in ntdll's
misc.c, so that I can use some ntdll internal functions, but maybe we
should have a separate file for this.

If any of you want to play around with this patch, Joshua Ashton (d9vk
creator) made a few patches on DXVK that implement D3D shared resources
using this extension and files, and  modified a D3D example to use shared
resources, which is what I've been using to test my implementation:

https://github.com/Joshua-Ashton/dxvk/tree/dxvk-shared-resources

D3D11 binary attached (Joshua didn't send me source)

Note: Make sure to reset to HEAD~1 then build, as I think the latest commit
is broken.  Also, replace
VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_(+KMT_)BIT with
VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_(+KMT_)BIT.  Also, this branch
only compiles with MSVC.  Alternatively, I'll attach the binaries to this
email.

Note 2: Make sure to use native d3dcompiler_47

Shared-Resources.tar.xz
https://drive.google.com/file/d/112N8xhnUJ0FyBDodgbP42fD31-VLq5ka/view?usp=drive_web

[Derek Lesho]

I just realized that I forgot the attach the patch itself!  My bad!

On Sun, Jul 28, 2019 at 9:06 PM Derek Lesho  wrote:
> Hello, before I break this patch up and submit a patchset, I'd like to inquire about a few things in this patch I have made.
>
> The biggest question I have is how we want to process allocate_info without modifying what is passed in.
>
> In order to copy the whole pNext chain or the unknown bits, we'd need a way to get the struct length given the sType.  To do this we'd have to generate something in make_vulkan, but I'm not sure of the specifics (mostly where the helper would go).
>
> Alternatively, we could try restoring the chain after it is finished, however this may not be thread safe, if the struct is being read in another thread.
>
> Additionally, I'd like to know how to better organize the functions for creating/operating on gpu-resource objects.  Right now they're in ntdll's misc.c, so that I can use some ntdll internal functions, but maybe we should have a separate file for this.
>
> If any of you want to play around with this patch, Joshua Ashton (d9vk creator) made a few patches on DXVK that implement D3D shared resources using this extension and files, and  modified a D3D example to use shared resources, which is what I've been using to test my implementation:
>
> https://github.com/Joshua-Ashton/dxvk/tree/dxvk-shared-resources
>
> D3D11 binary attached (Joshua didn't send me source)
>
> Note: Make sure to reset to HEAD~1 then build, as I think the latest commit is broken.  Also, replace VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_(+KMT_)BIT with VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_(+KMT_)BIT.  Also, this branch only compiles with MSVC.  Alternatively, I'll attach the binaries to this email.
>
> Note 2: Make sure to use native d3dcompiler_47
>
> Shared-Resources.tar.xz

[Marvin]

Hi,

While running your changed tests, I think I found new failures.
Being a bot and all I'm not very good at pattern recognition, so I might be
wrong, but could you please double-check?

Full results can be found at:
https://testbot.winehq.org/JobDetails.pl?Key=54897

Your paranoid android.

=== debian10 (build log) ===

../../../wine/dlls/winevulkan/vulkan.c:1219:5: error: unknown type name ‘VkImportMemoryFdInfoKHR’; did you mean ‘VkImportMemoryWin32HandleInfoKHR’?
../../../wine/dlls/winevulkan/vulkan.c:1233:19: error: request for member ‘fd’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1259:27: error: request for member ‘sType’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1259:36: error: ‘VK_STRUCTURE_TYPE_IMPORT_MEMORY_FD_INFO_KHR’ undeclared (first use in this function); did you mean ‘VK_STRUCTURE_TYPE_IMPORT_MEMORY_WIN32_HANDLE_INFO_KHR’?
../../../wine/dlls/winevulkan/vulkan.c:1260:27: error: request for member ‘pNext’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1261:27: error: request for member ‘handleType’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1283:75: error: request for member ‘fd’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1285:31: error: request for member ‘fd’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1292:35: error: request for member ‘fd’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1292:59: error: request for member ‘fd’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1322:17: error: unknown type name ‘VkMemoryGetFdInfoKHR’; did you mean ‘VkPresentInfoKHR’?
../../../wine/dlls/winevulkan/vulkan.c:1326:29: error: request for member ‘sType’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1326:38: error: ‘VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR’ undeclared (first use in this function); did you mean ‘VK_STRUCTURE_TYPE_SUBPASS_END_INFO_KHR’?
../../../wine/dlls/winevulkan/vulkan.c:1327:29: error: request for member ‘pNext’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1328:29: error: request for member ‘memory’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1329:29: error: request for member ‘handleType’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1331:35: error: ‘struct vulkan_device_funcs’ has no member named ‘p_vkGetMemoryFdKHR’; did you mean ‘p_vkFreeMemory’?
../../../wine/dlls/winevulkan/vulkan.c:1370:27: error: request for member ‘fd’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1371:33: error: request for member ‘fd’ in something not a structure or union
Task: The win32 build failed

=== debian10 (build log) ===

../../../wine/dlls/winevulkan/vulkan.c:1219:5: error: unknown type name ‘VkImportMemoryFdInfoKHR’; did you mean ‘VkImportMemoryWin32HandleInfoKHR’?
../../../wine/dlls/winevulkan/vulkan.c:1233:19: error: request for member ‘fd’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1259:27: error: request for member ‘sType’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1259:36: error: ‘VK_STRUCTURE_TYPE_IMPORT_MEMORY_FD_INFO_KHR’ undeclared (first use in this function); did you mean ‘VK_STRUCTURE_TYPE_IMPORT_MEMORY_WIN32_HANDLE_INFO_KHR’?
../../../wine/dlls/winevulkan/vulkan.c:1260:27: error: request for member ‘pNext’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1261:27: error: request for member ‘handleType’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1283:75: error: request for member ‘fd’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1285:31: error: request for member ‘fd’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1292:35: error: request for member ‘fd’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1292:59: error: request for member ‘fd’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1322:17: error: unknown type name ‘VkMemoryGetFdInfoKHR’; did you mean ‘VkPresentInfoKHR’?
../../../wine/dlls/winevulkan/vulkan.c:1326:29: error: request for member ‘sType’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1326:38: error: ‘VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR’ undeclared (first use in this function); did you mean ‘VK_STRUCTURE_TYPE_SUBPASS_END_INFO_KHR’?
../../../wine/dlls/winevulkan/vulkan.c:1327:29: error: request for member ‘pNext’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1328:29: error: request for member ‘memory’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1329:29: error: request for member ‘handleType’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1331:35: error: ‘struct vulkan_device_funcs’ has no member named ‘p_vkGetMemoryFdKHR’; did you mean ‘p_vkFreeMemory’?
../../../wine/dlls/winevulkan/vulkan.c:1370:27: error: request for member ‘fd’ in something not a structure or union
../../../wine/dlls/winevulkan/vulkan.c:1371:33: error: request for member ‘fd’ in something not a structure or union
Task: The wow64 build failed

[Derek Lesho]

Marvin, you've forgotten to run make_vulkan.

On Mon, Jul 29, 2019 at 12:17 AM Marvin  wrote:
> Hi,
>
> While running your changed tests, I think I found new failures.
> Being a bot and all I'm not very good at pattern recognition, so I might be
> wrong, but could you please double-check?
>
> Full results can be found at:
> https://testbot.winehq.org/JobDetails.pl?Key=54897
>
> Your paranoid android.
>
> === debian10 (build log) ===
>
> ../../../wine/dlls/winevulkan/vulkan.c:1219:5: error: unknown type name ‘VkImportMemoryFdInfoKHR’; did you mean ‘VkImportMemoryWin32HandleInfoKHR’?
> ../../../wine/dlls/winevulkan/vulkan.c:1233:19: error: request for member ‘fd’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1259:27: error: request for member ‘sType’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1259:36: error: ‘VK_STRUCTURE_TYPE_IMPORT_MEMORY_FD_INFO_KHR’ undeclared (first use in this function); did you mean ‘VK_STRUCTURE_TYPE_IMPORT_MEMORY_WIN32_HANDLE_INFO_KHR’?
> ../../../wine/dlls/winevulkan/vulkan.c:1260:27: error: request for member ‘pNext’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1261:27: error: request for member ‘handleType’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1283:75: error: request for member ‘fd’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1285:31: error: request for member ‘fd’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1292:35: error: request for member ‘fd’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1292:59: error: request for member ‘fd’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1322:17: error: unknown type name ‘VkMemoryGetFdInfoKHR’; did you mean ‘VkPresentInfoKHR’?
> ../../../wine/dlls/winevulkan/vulkan.c:1326:29: error: request for member ‘sType’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1326:38: error: ‘VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR’ undeclared (first use in this function); did you mean ‘VK_STRUCTURE_TYPE_SUBPASS_END_INFO_KHR’?
> ../../../wine/dlls/winevulkan/vulkan.c:1327:29: error: request for member ‘pNext’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1328:29: error: request for member ‘memory’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1329:29: error: request for member ‘handleType’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1331:35: error: ‘struct vulkan_device_funcs’ has no member named ‘p_vkGetMemoryFdKHR’; did you mean ‘p_vkFreeMemory’?
> ../../../wine/dlls/winevulkan/vulkan.c:1370:27: error: request for member ‘fd’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1371:33: error: request for member ‘fd’ in something not a structure or union
> Task: The win32 build failed
>
> === debian10 (build log) ===
>
> ../../../wine/dlls/winevulkan/vulkan.c:1219:5: error: unknown type name ‘VkImportMemoryFdInfoKHR’; did you mean ‘VkImportMemoryWin32HandleInfoKHR’?
> ../../../wine/dlls/winevulkan/vulkan.c:1233:19: error: request for member ‘fd’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1259:27: error: request for member ‘sType’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1259:36: error: ‘VK_STRUCTURE_TYPE_IMPORT_MEMORY_FD_INFO_KHR’ undeclared (first use in this function); did you mean ‘VK_STRUCTURE_TYPE_IMPORT_MEMORY_WIN32_HANDLE_INFO_KHR’?
> ../../../wine/dlls/winevulkan/vulkan.c:1260:27: error: request for member ‘pNext’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1261:27: error: request for member ‘handleType’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1283:75: error: request for member ‘fd’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1285:31: error: request for member ‘fd’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1292:35: error: request for member ‘fd’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1292:59: error: request for member ‘fd’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1322:17: error: unknown type name ‘VkMemoryGetFdInfoKHR’; did you mean ‘VkPresentInfoKHR’?
> ../../../wine/dlls/winevulkan/vulkan.c:1326:29: error: request for member ‘sType’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1326:38: error: ‘VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR’ undeclared (first use in this function); did you mean ‘VK_STRUCTURE_TYPE_SUBPASS_END_INFO_KHR’?
> ../../../wine/dlls/winevulkan/vulkan.c:1327:29: error: request for member ‘pNext’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1328:29: error: request for member ‘memory’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1329:29: error: request for member ‘handleType’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1331:35: error: ‘struct vulkan_device_funcs’ has no member named ‘p_vkGetMemoryFdKHR’; did you mean ‘p_vkFreeMemory’?
> ../../../wine/dlls/winevulkan/vulkan.c:1370:27: error: request for member ‘fd’ in something not a structure or union
> ../../../wine/dlls/winevulkan/vulkan.c:1371:33: error: request for member ‘fd’ in something not a structure or union
> Task: The wow64 build failed

[PATCH] testbot: Rerun make_vulkan when it is patched.

[Francois Gouget]

Signed-off-by: Francois Gouget 
---
 testbot/lib/Build/Utils.pm            | 10 ++++++++++
 testbot/lib/WineTestBot/PatchUtils.pm |  9 ++++++++-
 2 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/testbot/lib/Build/Utils.pm b/testbot/lib/Build/Utils.pm
index 4c3538e4..f5b66a26 100644
--- a/testbot/lib/Build/Utils.pm
+++ b/testbot/lib/Build/Utils.pm
@@ -126,6 +126,16 @@ sub ApplyPatch($$)
       return undef;
     }
   }
+  if ($Impacts->{MakeVulkan})
+  {
+    InfoMsg "\nRunning make_vulkan\n";
+    system("cd '$DataDir/$Dir' && set -x && ./dlls/winevulkan/make_vulkan");
+    if ($? != 0)
+    {
+      LogMsg "make_vulkan failed\n";
+      return undef;
+    }
+  }

if ($Impacts->{MakeMakefiles})
   {
diff --git a/testbot/lib/WineTestBot/PatchUtils.pm b/testbot/lib/WineTestBot/PatchUtils.pm
index b3dcd51f..ba1add75 100644
--- a/testbot/lib/WineTestBot/PatchUtils.pm
+++ b/testbot/lib/WineTestBot/PatchUtils.pm
@@ -325,6 +325,11 @@ sub GetPatchImpacts($)
       $Impacts->{PatchedRoot} = $Impacts->{MakeRequests} = 1;
       $Impacts->{IsWinePatch} = 1;
     }
+    elsif ($Line =~ m=^--- \w+/dlls/winevulkan/make_vulkan$=)
+    {
+      $Impacts->{PatchedRoot} = $Impacts->{MakeVulkan} = 1;
+      $Impacts->{IsWinePatch} = 1;
+    }
     elsif ($Line =~ m=^--- /dev/null$=)
     {
       $Change = "new";
@@ -370,7 +375,8 @@ sub GetPatchImpacts($)

# Also backup the build-related fields.
       foreach my $Field ("Autoconf", "MakeMakefiles", "MakeRequests",
-                         "PatchedRoot", "PatchedModules", "PatchedTests")
+                         "MakeVulkan", "PatchedRoot", "PatchedModules",
+                         "PatchedTests")
       {
         $PastImpacts->{$Field} = $Impacts->{$Field};
         $Impacts->{$Field} = undef;
@@ -465,6 +471,7 @@ sub GetPatchImpacts($)
       $Impacts->{Autoconf} ||= $PastImpacts->{Autoconf};
       $Impacts->{MakeMakefiles} ||= $PastImpacts->{MakeMakefiles};
       $Impacts->{MakeRequests} ||= $PastImpacts->{MakeRequests};
+      $Impacts->{MakeVulkan} ||= $PastImpacts->{MakeVulkan};
       $Impacts->{RebuildRoot} ||= $PastImpacts->{PatchedRoot};
       $Impacts->{RebuildModules} ||= $PastImpacts->{PatchedModules};
       map { $Impacts->{BuildModules}->{$_} = 1 } keys %{$PastImpacts->{BuildModules}};