0

I am trying to encrypt some text using OpenSSL's RSA encryption functions. My main issue is that the length of the encrypted RSA text varies between 0 and 256.

My RSA encryption function is:

/* Encrypt data using RSA */
char* rsa_encrypt(char* pub_key_filename, const unsigned char *data)
{
   int padding = RSA_PKCS1_PADDING;

   FILE *fp_pub;
   fp_pub = fopen(pub_key_filename, "rb");

   if (fp_pub == NULL)
   {
      printf("There was an error opening the public key file. Exiting!\n");
      exit(EXIT_FAILURE);
   }

   RSA *pub_key = PEM_read_RSA_PUBKEY(fp_pub, NULL, NULL, NULL);

   char *encrypted = malloc(2048);
   int i;
   for (i = 0; i < (2048); i++)
   {
      encrypted[i] = '\0';
   }

   int result = RSA_public_encrypt(strlen(data), data, encrypted, pub_key, padding);
   if (result == -1)
   {
      printf("There was an error during RSA encryption.\n");
      return "ERROR_RSA_ENCRYPTION";
   }

   fclose(fp_pub);

   return encrypted;
}

The following code involves trying to encrypt some text:

const unsigned char *key = (unsigned char *)"abcdefghijklmnopqrstuvwxyzabcdef";
unsigned char *encrypted_aes_key = rsa_encrypt("public.pem", key);

I know that RSA with no padding is primitive RSA encryption and the resulting length is between 0 and n (RSA bit size) as seen here but my code is using RSA_PKCS1_PADDING so I am not sure why I am still getting variable length output.

Community
  • 1
  • 1

1 Answers1

1

The length of the encrypted data is returned in result from:

int result = RSA_public_encrypt(strlen(data), data, encrypted, pub_key, padding);

The encrypted data returned in encrypted is binary data. You can't do a strlen on it. The data is not 0 terminated and might contain some random 0 in it.

Ebbe M. Pedersen
  • 7,250
  • 3
  • 27
  • 47
  • Thank you. I decided to use the OpenSSL BIO functions such as BIO_f_base64() to base64 encode the binary RSA output for further use. –  Mar 28 '16 at 20:10