[PATCH v3 0/1] MR6221: rsaenh: Don't crash when decrypting empty strings

[Jakub Petržílka (＠kubapet)]

Some applications apparently attempt to decrypt empty strings in some edge cases which currently leads to a page fault.

Wine-Bug https://bugs.winehq.org/show_bug.cgi?id=57042

--
  v3: rsaenh: Don't crash when decrypting empty strings

https://gitlab.winehq.org/wine/wine/-/merge_requests/6221

[PATCH v3 1/1] rsaenh: Don't crash when decrypting empty strings

[Jakub Petrzilka]

From: Jakub Petrzilka

Some applications apparently attempts to decrypt empty strings in some edge cases which currently lead to page fault.

Wine-Bug https://bugs.winehq.org/show_bug.cgi?id=57042
---
 dlls/advapi32/tests/crypt.c | 4 ++++
 dlls/rsaenh/rsaenh.c        | 6 ++++++
 2 files changed, 10 insertions(+)

diff --git a/dlls/advapi32/tests/crypt.c b/dlls/advapi32/tests/crypt.c
index 8b25cb24f16..dfdcd0ce5ad 100644
--- a/dlls/advapi32/tests/crypt.c
+++ b/dlls/advapi32/tests/crypt.c
@@ -315,6 +315,10 @@ static void test_incorrect_api_usage(void)
     result = CryptGenKey(0, CALG_RC4, 0, &hKey);
     ok (!result && GetLastError() == ERROR_INVALID_PARAMETER, "%ld\n", GetLastError());

+    dwLen = 0;
+    SetLastError(0xdeadbeef);
+    result = CryptDecrypt(hKey, 0, TRUE, 0, &temp, &dwLen);
+    ok (!result && GetLastError() == NTE_BAD_LEN, "%lx\n", GetLastError());
     dwLen = 1;
     result = CryptDecrypt(hKey, 0, TRUE, 0, &temp, &dwLen);
     ok (result, "%ld\n", GetLastError());
diff --git a/dlls/rsaenh/rsaenh.c b/dlls/rsaenh/rsaenh.c
index f854e35dfdf..7440aba7200 100644
--- a/dlls/rsaenh/rsaenh.c
+++ b/dlls/rsaenh/rsaenh.c
@@ -2763,6 +2763,12 @@ BOOL WINAPI RSAENH_CPDecrypt(HCRYPTPROV hProv, HCRYPTKEY hKey, HCRYPTHASH hHash,
         return FALSE;
     }

+    if (!*pdwDataLen)
+    {
+        SetLastError(NTE_BAD_LEN);
+        return FALSE;
+    }
+
     dwMax=*pdwDataLen;

if (GET_ALG_TYPE(pCryptKey->aiAlgid) == ALG_TYPE_BLOCK) {

[PATCH v3 1/1] rsaenh: Don't crash when decrypting empty strings

[Marvin]

Hi,

It looks like your patch introduced the new failures shown below.
Please investigate and fix them before resubmitting your patch.
If they are not new, fixing them anyway would help a lot. Otherwise
please ask for the known failures list to be updated.

The tests also ran into some preexisting test failures. If you know how
to fix them that would be helpful. See the TestBot job for the details:

The full results can be found at:
https://testbot.winehq.org/JobDetails.pl?Key=147593

Your paranoid android.

=== debian11b (64 bit WoW report) ===

kernel32:
comm.c:1586: Test failed: Unexpected time 1001, expected around 500

ws2_32:
protocol.c:1657: Test failed: Wrong address data
protocol.c:1657: Test failed: Wrong address data

[Jakub Petržílka (＠kubapet)]

Thanks! it indeed does fix it. So I've updated the MR with the diff you sent me.

[PATCH v3 0/1] MR6221: rsaenh: Don't crash when decrypting empty strings - approved

[Hans Leidekker (＠hans)]

This merge request was approved by Hans Leidekker.