2

I've been using openssl library in my C++ project recently, and I'm facing an issue that i can't fix by myself.

I'm actually trying to load an RSA public key stored in a file and encrypt 64 bytes. My code works when it uses a public key generated using the function RSA_generate_key, but when i'm using my own public key, it won't work anymore for some reason.

I've suspected key format from pkcs1 pkcs8, tried both PEM_read_RSAPublicKey and PEM_read_RSA_PUBKEY, still not working for some reason...

This is my public key :

-----BEGIN RSA PUBLIC KEY-----
MEYCQQDE91cW7INdIyVon5H/he2b/DIR25wWT0GFLiZOVp0oAgCAVKDvRZ5+Pqu4
f65XbnNUNNHRJLMLEb1t4JgUhgFVAgER
-----END RSA PUBLIC KEY-----

The key from RSA_generate_key function from Openssl library, which is working :

-----BEGIN RSA PUBLIC KEY-----
MEYCQQDsg/4Qm153/Pr8JRruC0SnVvTrWg/lIPheezIpkwVeWjNz9lMDXNUjdK8v
QgfNUCRJYbnxYIeruAdwTzS/bDXbAgER
-----END RSA PUBLIC KEY-----

And here's my code :

RSA.h :

#include <iostream>
#include <openssl/rsa.h>
#include <openssl/pem.h>
#include <openssl/err.h>
#include <string>

#ifndef RSA_ALGORITHM_H
#define RSA_ALGORITHM_H

#define KEY_LENGTH       512
#define PUBLIC_EXPONENT  17
#define PUBLIC_KEY_PEM   1
#define PRIVATE_KEY_PEM  0

#define LOG(x)               \
        std::cout << x << std::endl;   \

 /*
  * @brief   create_RSA function creates public key and private key file
  *
  */
RSA* create_RSA(RSA* keypair, int pem_type, char* file_name);

/*
 * @brief   public_ecrypt function encrypts data.
 * @return  If It is fail, return -1
 */
int public_encrypt(int flen, unsigned char* from, unsigned char* to, RSA* key, int padding);

/*
 * @brief   private_decrypt function decrypt data.
 * @return  If It is fail, return -1
 */
int private_decrypt(int flen, unsigned char* from, unsigned char* to, RSA* key, int padding);

/*
 * @brief   create_ecrypted_file function creates .bin file. It contains encrypted data.
 */
void create_encrypted_file(char* encrypted, RSA* key_pair);

#endif //RSA_ALGORITHM_H

RSA.cpp :

#include "RSA.h"

#include <iostream>
#include <string.h>
#include <openssl/rsa.h>
#include <openssl/pem.h>
#include <openssl/err.h>
#include <sstream>
#include <iomanip>

int public_encrypt(int flen, unsigned char* from, unsigned char* to, RSA* key, int padding) {
    int result = RSA_public_encrypt(flen, from, to, key, padding);
    return result;
}

void create_encrypted_file(char* encrypted, RSA* key_pair) {

    FILE* encrypted_file = fopen("encrypted_file.bin", "w");
    fwrite(encrypted, sizeof(*encrypted), RSA_size(key_pair), encrypted_file);
    fclose(encrypted_file);
}

RSA* createRSA(int pem_type, char* file_name) {

    RSA* rsa = NULL;
    FILE* fp = NULL;

    if (pem_type == PUBLIC_KEY_PEM) {

        fp = fopen(file_name, "rb");
        PEM_read_RSAPublicKey(fp, &rsa, NULL, NULL);
        fclose(fp);

    }
    else if (pem_type == PRIVATE_KEY_PEM) {

        fp = fopen(file_name, "rb");
        PEM_read_RSAPrivateKey(fp, &rsa, NULL, NULL);
        fclose(fp);

    }

    return rsa;
}

int main() {

    LOG("RSA has been started.");

    char public_key_pem[11] = "public_key";

    RSA* public_key = createRSA(PUBLIC_KEY_PEM, public_key_pem);
    LOG("Public key pem file has been created.");;

    char message[KEY_LENGTH] = "\xc8\xcd\x21\x74\xb9\x84\x33\xb9\x30\x94\xb3\x60\x26\xde\x12\x5a\x7f\x5e\xd8\x5e\xc2\x7e\xe6\xbb\x9e\x99\x6c\xb3\xb9\x38\xe9\xc6\x23\x8c\xc6\x5d\x36\x15\xfb\x63\x5f\x6f\x08\x0f\x6d\xda\x06\x31\x59\x28\xbc\xae\x4c\xcf\x80\x2f\x96\x80\x54\x7d\xb5\x7b\x82\x83";
    char* encrypt = NULL;


    LOG(KEY_LENGTH);
    LOG(PUBLIC_EXPONENT);

    encrypt = (char*)malloc(RSA_size(public_key));
    int encrypt_length = public_encrypt(RSA_size(public_key), (unsigned char*)message, (unsigned char*)encrypt, public_key, RSA_NO_PADDING);
    if (encrypt_length == -1) {
        LOG("An error occurred in public_encrypt() method");
    }
    LOG("Data has been encrypted.");

    create_encrypted_file(encrypt, public_key);
    LOG("Encrypted file has been created.");

    free(public_key);
    free(encrypt);
    LOG("RSA has been finished.");

    return 0;
}

I've seen many posts and haven't found any fix, even though that this one was extremely similar to my issue

. Load public key to create rsa object for public encryption

tortank
  • 23
  • 3
  • For what it's worth I can't load that key using the `openssl` binary either. Are you sure that's a valid key? How did you generate it? – tadman Apr 01 '20 at 21:35
  • @tadman The key have been stripped from a certificate. – tortank Apr 01 '20 at 21:37

2 Answers2

0

The error is actually mathematically related. Neither the key nor the the code is causing the issue. If you change the key or the data to encrypt, it'll work fine for some reason. So i guess something is happening during RSA calculation, but no smart enough to figure out what.

tortank
  • 23
  • 3
0

Both keys are public RSA keys with a size of 512 bits and an exponent of 17, specified in PKCS1-PEM format.

The message m, the modulus n_fail of the not working key and the modulus n_ok of the working key are:

m =      0xc8cd2174b98433b93094b36026de125a7f5ed85ec27ee6bb9e996cb3b938e9c6238cc65d3615fb635f6f080f6dda06315928bcae4ccf802f9680547db57b8283
n_fail = 0xc4f75716ec835d2325689f91ff85ed9bfc3211db9c164f41852e264e569d2802008054a0ef459e7e3eabb87fae576e735434d1d124b30b11bd6de09814860155
n_ok =   0xec83fe109b5e77fcfafc251aee0b44a756f4eb5a0fe520f85e7b322993055e5a3373f653035cd52374af2f4207cd50244961b9f16087abb807704f34bf6c35db

A comparison shows that

n_fail < m < n_ok

For RSA the condition m < n must apply. This condition is violated for n_fail, which is the cause of the issue. This also means that the corresponding key itself is not invalid. It can be used to encrypt messages that do not violate m < n. For the posted message, however, its modulus is too small.

Regarding security: Nowadays the key should have a size of 2048 bits and one of the paddings specified in RFC8017 (RSAES-PKCS1-v1_5 or RSAES-OAEP) should be used.

Community
  • 1
  • 1
Topaco
  • 40,594
  • 4
  • 35
  • 62
  • Following your answer, I made a python script with same values and encryption worked even though that it was n < m instead of m < n https://stackoverflow.com/questions/60997745/trouble-loading-rsa-public-key-from-file-2 – tortank Apr 02 '20 at 18:07
  • If a message is encrypted and then decrypted again and the condition `m < n` is violated, the original and decrypted message will generally be different. You can easily check this with small numbers, e.g. with the numbers from the Wikipedia article [RSA](https://en.wikipedia.org/wiki/RSA_(cryptosystem)#Example). Also note that you are using different implementations. Obviously the C implementation returns an error for the case `m >= n`, your Python implementation (possibly) does not. In the latter case, the error is generally revealed during decryption (which will fail). – Topaco Apr 02 '20 at 21:12
  • Decryption worked and gave me exact same value as before encryption. But, can you describe me what makes those two implementations different from each other? – tortank Apr 02 '20 at 21:18
  • I suspect there's a check in the C code somewhere to see if `m < n`. You should also post the private key (if possible), otherwise others cannot reproduce the decryption. – Topaco Apr 02 '20 at 21:27
  • As your own example shows, the decryption does not return the original message. You have to add `n` first to get the original message (see line `m = m + n # Had to put this in the way to recover correct data`). Also, the term to be added is not always `n`, but depends on `m`. This means that the uniqueness is lost. How should the recipient know if nothing, `n`, `2n`... should be added? This problem does not exist for `m < n`. Nothing needs to be added here. – Topaco Apr 02 '20 at 22:37
  • That still doesn't explain why openssl can't encrypt the data unlike python does. – tortank Apr 02 '20 at 22:47
  • I've already given you a plausible assumption regarding OpenSSL: OpenSSL just doesn't seem to allow this forbidden case, which is quite reasonable. But in order to be able to answer this _definitely_, you need to look at the source code. – Topaco Apr 02 '20 at 23:07