[PATCH v2 0/2] MR6221: rsaenh: Don't crash when decrypting empty strings

[Jakub Petržílka (＠kubapet)]

Some applications apparently attempt to decrypt empty strings in some edge cases which currently leads to a page fault.

Wine-Bug https://bugs.winehq.org/show_bug.cgi?id=57042

--
  v2: rsaenh: Don't crash when decrypting empty strings

https://gitlab.winehq.org/wine/wine/-/merge_requests/6221

[PATCH v2 1/2] rsaenh: Don't crash when decrypting empty strings

[Jakub Petrzilka]

From: Jakub Petrzilka

Some applications apparently attempts to decrypt empty strings in some edge cases which currently lead to page fault.

Wine-Bug https://bugs.winehq.org/show_bug.cgi?id=57042
---
 dlls/rsaenh/rsaenh.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dlls/rsaenh/rsaenh.c b/dlls/rsaenh/rsaenh.c
index f854e35dfdf..25aba026152 100644
--- a/dlls/rsaenh/rsaenh.c
+++ b/dlls/rsaenh/rsaenh.c
@@ -2798,7 +2798,8 @@ BOOL WINAPI RSAENH_CPDecrypt(HCRYPTPROV hProv, HCRYPTKEY hKey, HCRYPTHASH hHash,
             memcpy(in, out, pCryptKey->dwBlockLen);
         }
         if (Final) {
-            if (pbData[*pdwDataLen-1] &&
+            if (*pdwDataLen >=1 &&
+             pbData[*pdwDataLen-1] &&
              pbData[*pdwDataLen-1] <= pCryptKey->dwBlockLen &&
              pbData[*pdwDataLen-1] <= *pdwDataLen) {
                 BOOL padOkay = TRUE;

[PATCH v2 2/2] rsaenh: Don't crash when decrypting empty strings

[Jakub Petrzilka]

From: Jakub Petrzilka

Some applications apparently attempts to decrypt empty strings in some edge cases which currently lead to page fault.

Wine-Bug https://bugs.winehq.org/show_bug.cgi?id=57042
---
 dlls/advapi32/tests/crypt.c | 4 ++++
 dlls/rsaenh/rsaenh.c        | 8 +++++++-
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/dlls/advapi32/tests/crypt.c b/dlls/advapi32/tests/crypt.c
index 8b25cb24f16..dfdcd0ce5ad 100644
--- a/dlls/advapi32/tests/crypt.c
+++ b/dlls/advapi32/tests/crypt.c
@@ -315,6 +315,10 @@ static void test_incorrect_api_usage(void)
     result = CryptGenKey(0, CALG_RC4, 0, &hKey);
     ok (!result && GetLastError() == ERROR_INVALID_PARAMETER, "%ld\n", GetLastError());

+    dwLen = 0;
+    SetLastError(0xdeadbeef);
+    result = CryptDecrypt(hKey, 0, TRUE, 0, &temp, &dwLen);
+    ok (!result && GetLastError() == NTE_BAD_LEN, "%lx\n", GetLastError());
     dwLen = 1;
     result = CryptDecrypt(hKey, 0, TRUE, 0, &temp, &dwLen);
     ok (result, "%ld\n", GetLastError());
diff --git a/dlls/rsaenh/rsaenh.c b/dlls/rsaenh/rsaenh.c
index 25aba026152..f091f0f098b 100644
--- a/dlls/rsaenh/rsaenh.c
+++ b/dlls/rsaenh/rsaenh.c
@@ -2763,6 +2763,12 @@ BOOL WINAPI RSAENH_CPDecrypt(HCRYPTPROV hProv, HCRYPTKEY hKey, HCRYPTHASH hHash,
         return FALSE;
     }

+    if (!*pdwDataLen)
+    {
+        SetLastError(NTE_BAD_LEN);
+        return FALSE;
+    }
+
     dwMax=*pdwDataLen;

if (GET_ALG_TYPE(pCryptKey->aiAlgid) == ALG_TYPE_BLOCK) {
@@ -2798,7 +2804,7 @@ BOOL WINAPI RSAENH_CPDecrypt(HCRYPTPROV hProv, HCRYPTKEY hKey, HCRYPTHASH hHash,
             memcpy(in, out, pCryptKey->dwBlockLen);
         }
         if (Final) {
-            if (*pdwDataLen >=1 &&
+            if (
              pbData[*pdwDataLen-1] &&
              pbData[*pdwDataLen-1] <= pCryptKey->dwBlockLen &&
              pbData[*pdwDataLen-1] <= *pdwDataLen) {

[PATCH v2 2/2] rsaenh: Don't crash when decrypting empty strings

[Marvin]

Hi,

It looks like your patch introduced the new failures shown below.
Please investigate and fix them before resubmitting your patch.
If they are not new, fixing them anyway would help a lot. Otherwise
please ask for the known failures list to be updated.

The tests also ran into some preexisting test failures. If you know how
to fix them that would be helpful. See the TestBot job for the details:

The full results can be found at:
https://testbot.winehq.org/JobDetails.pl?Key=147591

Your paranoid android.

=== debian11b (64 bit WoW report) ===

dinput:
hotplug.c:194: Test failed: 0x500: GetDeviceState returned 0
hid.c:720: Test failed: CreateFile failed, error 3

hid:
device.c:123: Test failed: Failed to open L"\\?\hid#vid_1209&pid_0001#0&0000&0#{4d1e55b2-f16f-11cf-88cb-001111000030}", error 3.
device.c:123: Test failed: Failed to open L"\\?\hid#vid_1209&pid_0001&col01#256&wine test&0&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}", error 3.
device.c:123: Test failed: Failed to open L"\\?\hid#vid_1209&pid_0001&col02#256&wine test&0&0&0001#{4d1e55b2-f16f-11cf-88cb-001111000030}", error 3.
device.c:123: Test failed: Failed to open L"\\?\hid#vid_845e&pid_0001#0&0000&0&0&0#{4d1e55b2-f16f-11cf-88cb-001111000030}", error 3.
device.c:123: Test failed: Failed to open L"\\?\hid#vid_845e&pid_0002#0&0000&0&0&0#{4d1e55b2-f16f-11cf-88cb-001111000030}", error 3.
device.c:170: Test failed: got error 3
device.c:173: Test failed: Failed to get preparsed data(0x6)
device.c:175: Test failed: Failed to get Caps(0xc0110001)
device.c:170: Test failed: got error 3
device.c:173: Test failed: Failed to get preparsed data(0x6)
device.c:175: Test failed: Failed to get Caps(0xc0110001)
device.c:170: Test failed: got error 3
device.c:173: Test failed: Failed to get preparsed data(0x6)
device.c:175: Test failed: Failed to get Caps(0xc0110001)
device.c:170: Test failed: got error 3
device.c:173: Test failed: Failed to get preparsed data(0x6)
device.c:175: Test failed: Failed to get Caps(0xc0110001)
device.c:170: Test failed: got error 3
device.c:173: Test failed: Failed to get preparsed data(0x6)
device.c:175: Test failed: Failed to get Caps(0xc0110001)
device.c:170: Test failed: got error 3
device.c:173: Test failed: Failed to get preparsed data(0x6)
device.c:175: Test failed: Failed to get Caps(0xc0110001)
device.c:319: Test failed: Failed to get product string(0x6)
device.c:323: Test failed: Failed to get preparsed data(0x6)
device.c:325: Test failed: Failed to get Caps(0xc0110001)
device.c:170: Test failed: got error 3
device.c:173: Test failed: Failed to get preparsed data(0x6)
device.c:175: Test failed: Failed to get Caps(0xc0110001)
device.c:170: Test failed: got error 3
device.c:173: Test failed: Failed to get preparsed data(0x6)
device.c:175: Test failed: Failed to get Caps(0xc0110001)
device.c:170: Test failed: got error 3
device.c:173: Test failed: Failed to get preparsed data(0x6)
device.c:175: Test failed: Failed to get Caps(0xc0110001)
device.c:170: Test failed: got error 3
device.c:173: Test failed: Failed to get preparsed data(0x6)
device.c:175: Test failed: Failed to get Caps(0xc0110001)
device.c:170: Test failed: got error 3
device.c:173: Test failed: Failed to get preparsed data(0x6)
device.c:175: Test failed: Failed to get Caps(0xc0110001)
device.c:170: Test failed: got error 3
device.c:173: Test failed: Failed to get preparsed data(0x6)
device.c:175: Test failed: Failed to get Caps(0xc0110001)
device.c:405: Test failed: Failed to get product string(0x6)
device.c:409: Test failed: Failed to get preparsed data(0x6)
device.c:411: Test failed: Failed to get Caps(0xc0110001)

user32:
input.c:2206: Test failed: got 0 messages
input.c:2216: Test failed: got 0 messages
input.c:1933: Test failed: expected non-zero
input.c:1939: Test failed: expected -1, got 0
input.c:1940: Test failed: expected 122, got -559038737
input.c:1941: Test failed: expected non-zero
input.c:1945: Test failed: expected non-zero
input.c:2080: Test failed: expected non-zero