On Thu, 2021-07-08 at 14:25 +0200, Hans Leidekker wrote:
/* CNG_RSA_PUBLIC_KEY_BLOB always stores the exponent and modulus
* in big-endian format, so we need to convert from the native
* endianness
*/
CRYPT_memcpy_be((BYTE *)pvStructInfo + sizeof(BCRYPT_RSAKEY_BLOB),
(BYTE *)&decodedKey->pubexp, sizeof(DWORD));
CRYPT_memcpy_be((BYTE *)pvStructInfo + sizeof(BCRYPT_RSAKEY_BLOB) +
sizeof(DWORD), decodedKey->modulus.pbData,
decodedKey->modulus.cbData);
Exponent and modulus are handled as a byte arrays so native endianess is not relevant there. They should be reversed unconditionally here as well as in the encoder and the tests.
Though in the test you use a DWORD for the exponent, so you would need to reverse that depending on host endianness, or you could store it as a byte array.