Unable to decrypt data encoded via OpenSSL on iPhone

Question

Kind of a followup to my previous question to: How do I get the initialization vector (iv) from OpenSSL encrypted data

I'm using OpenSSLcommand line utility to encrypt a string and then attempting to use <CommonCrypto/CommonCryptor.h> to decrypt the string on an iPhone. Using Dropbox SDK, an xml file with encrypted strings is loaded onto the iPhone where my app attempts to parse and decrypt strings within this file.

Here's an example of the openssl command:

printf %s "Hello" | openssl enc -aes-128-cbc -K 00ff349830193845af43984758690213 -iv 0 -base64

The above base 64 string is placed in an XML file which is then parsed by the app.

I'm using Matt Gallagher's NSData addition to decode the base64 text. I'm assuming that's working correctly; I haven't really found a good way to test it. (Source: http://cocoawithlove.com/2009/06/base64-encoding-options-on-mac-and.html).

Here's the method to decode the encrypted string.
The key is an NSString in this case equal to @"00ff349830193845af43984758690213".

+ (NSString *)string:(NSString *)encryptedString withAES128Key:(NSString *)key {

// decode base64, from Matt Gallagher's NSData category
NSData *b64DecodedData = [NSData dataFromBase64String:encryptedString];

NSData *keyData = [key dataUsingEncoding:NSUTF8StringEncoding];

// fyi, I plan to replace this later with a random iv
NSData *ivData = [@"00000000000000000000000000000000" dataUsingEncoding:NSUTF8StringEncoding];

// decrypt the string
NSData *decodedData = [self doCipher:b64DecodedData iv:ivData key:keyData context:kCCDecrypt];

NSString *unencryptedString = [[NSString alloc] initWithBytes:[decodedData bytes] length:[decodedData length] encoding:NSUTF8StringEncoding];

return [unencryptedString autorelease];
}

Here's the method that does the actual decrypting: (Credit for this method goes to a fellow stackoverflow user.)

+ (NSData *)doCipher:(NSData *)dataIn
              iv:(NSData *)iv
             key:(NSData *)symmetricKey
         context:(CCOperation)encryptOrDecrypt
{
CCCryptorStatus ccStatus   = kCCSuccess;
size_t          cryptBytes = 0;    // Number of bytes moved to buffer.
NSMutableData  *dataOut    = [NSMutableData dataWithLength:dataIn.length + kCCBlockSizeAES128];

ccStatus = CCCrypt( encryptOrDecrypt,
                   kCCAlgorithmAES128,
                   kCCOptionPKCS7Padding,
                   [symmetricKey bytes], 
                   kCCKeySizeAES128,
                   iv,
                   dataIn.bytes,
                   dataIn.length,
                   dataOut.mutableBytes,
                   dataOut.length,
                   &cryptBytes);

// error occurs here, error -4304 kCCDecodeError
if (ccStatus != kCCSuccess) {
    // Handle error
    NSLog(@"CCCrypt status: %d", ccStatus);
}

dataOut.length = cryptBytes;

return dataOut;
}

An error occurs, error code -4304 which is kCCDecodeError because ccStatus is not equal to kCCSuccess.

I feel the key and iv are not being set as NSData objects correctly. OpenSSL requires the key and iv to be hex values, which I have done and careful to set them to exactly 128 bits. However, I think I'm missing something in converting those strings to NSData for the doCipher method.

Any help is greatly appreciated! Been toying with this all day.

zaph · Accepted Answer · 2011-09-14T19:53:23.847

While the iv is handles incorrectly that is the least of the problems.

A decode error sounds like incorrect argument lengths since any random iv, key and data should be valid input. (my wife agrees and she does this stuff professionally.) Check things like the key and data length after converting them to NSData. Note that passing encrypted data with an incorrect or incompatible padding will also result in a decoding error.

Write a test for Base64, your iOS code vs openssl.

Work up to the solution from simpler tests.

For example drop the base64 until you get the encryption top work. Try simple data, say one block length of 0's, padding can be a problem. Try a simpler key such as all 0's. You can use OPENSSL on the Mac Terminal command line.

Once the basic encryption is working add back in the needed functionality.

For openssl from the command line use input and output files, they will handle binary so you will not have that hurdle at least initially. Here is a sample:

(file_orig.txt contains: "1234567890123456")

openssl enc -e -aes-128-cbc -K 00ff349830193845af43984758690213 -p -iv 0 -nosalt -in file_orig.txt -out file_aes.txt

which prints out the key it generated as well as the iv it used:

key=00ff349830193845af43984758690213
iv =00000000000000000000000000000000

Then you can read the same data files in your iOS method.

Here is an iOS method that uses the files openssl creates:
(put the key openssl output into the file key-hex-openssl.txt)

NSData *keyHexData = [@"00ff349830193845af43984758690213" dataUsingEncoding:NSUTF8StringEncoding];
NSData *testData   = [NSData dataWithContentsOfFile:@"yourDirectoryPath/file_aes.txt"];
NSData *clearData  = [NSData dataWithContentsOfFile:@"yourDirectoryPath/file_orig.txt"];

NSLog(@"keyHexData: %@", keyHexData);
NSLog(@"testData:   %@", testData);
NSLog(@"clearData:  %@", clearData);

unsigned char keyBytes[16];
unsigned char *hex = (uint8_t *)keyHexData.bytes;

char byte_chars[3] = {'\0','\0','\0'};
for (int i=0; i<16; i++) {
    byte_chars[0] = hex[i*2];
    byte_chars[1] = hex[(i*2)+1];
    keyBytes[i] = strtol(byte_chars, NULL, 16);
}
NSData *keyData = [NSData dataWithBytes:keyBytes length:16];
NSLog(@"keyData:    %@", keyData);

NSData *ivData = [NSData dataWithBytes:(char []){0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0} length:16];
NSLog(@"ivData:     %@", ivData);

CCCryptorStatus ccStatus   = kCCSuccess;
size_t          cryptBytes = 0;    // Number of bytes moved to buffer.
NSMutableData  *clearOut   = [NSMutableData dataWithLength:testData.length];

ccStatus = CCCrypt(kCCDecrypt,
                   kCCAlgorithmAES128,
                   kCCOptionPKCS7Padding,
                   keyData.bytes, 
                   kCCKeySizeAES128,
                   ivData.bytes,
                   testData.bytes,
                   testData.length,
                   clearOut.mutableBytes,
                   clearOut.length,
                   &cryptBytes);

if (ccStatus != kCCSuccess) {
    NSLog(@"CCCrypt status: %d", ccStatus);
}

clearOut.length = cryptBytes;
NSLog(@"clearOut:   %@", clearOut);
keyHexData: <41393641 34344436 31343245 43463546 33444339 30303038 46453941 34383838>
testData:   <86a8b306 0f33db02 01e77e66 af5bcb3a>
clearData:  <31323334 35363738 39303132 33343536>
keyData:    <a96a44d6 142ecf5f 3dc90008 fe9a4888>
ivData:     <00000000 00000000 00000000 00000000>
clearOut:   <31323334 35363738 39303132 33343536>

Note that clearData has been recovered into clearOut

This demonstrates encrypting with openssl and decrypting with CommonCrypto.

Problems to be overcome:
1) Base64 needs to be added

This is a starting point to complete the encryption needed.

It turns out that PKCS5 and PKCS7 use the same algorithm so for padding openssl and ComonCrypto are compatible. — zaph, Sep 14 '11 at 19:54
Funny how many people who just use CommonCrypto don't really know that the key sent to CCCrypt should be raw bytes. Thanks for this answer. — Tudor, Feb 28 '12 at 16:38

indiv · Answer 2 · 2011-09-14T19:52:07.940

You're providing CCCrypt the UTF8 representation of your key and IV when the function wants raw bytes. If you are storing your key and IV in string form, where the string is the hexadecimal representation of the bytes, then you need to convert this string of hex digits back to raw bytes.

For example, let's take your IV of all zero bytes.

You provided OpenSSL with the IV "00000000000000000000000000000000". OpenSSL takes that string and converts each two hex digits into their respective byte, and comes up with these 16 bytes:

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  (This is an array of 16 zero bytes)

You, however, take the IV string and get the UTF8 representation of it.

// fyi, I plan to replace this later with a random iv
NSData *ivData = [@"00000000000000000000000000000000" dataUsingEncoding:NSUTF8StringEncoding];

What you end up with are these bytes:

30 30 30 30 30 30 30 30 ...

Because the character '0' in UTF8 is 0x30. So convert your hex representations into bytes, not UTF8 characters, and your key and IV will match OpenSSL's.

OpenSSL has a function called set_hex that does the conversion from string to bytes (which in C are held in an unsigned char array).

#define BIO_printf fprintf
#define bio_err stderr
int set_hex(char *in, unsigned char *out, int size)
        {
        int i,n;
        unsigned char j;

        n=strlen(in);
        if (n > (size*2))
                {
                BIO_printf(bio_err,"hex string is too long\n");
                return(0);
                }
        memset(out,0,size);
        for (i=0; i<n; i++)
                {
                j=(unsigned char)*in;
                *(in++)='\0';
                if (j == 0) break;
                if ((j >= '0') && (j <= '9'))
                        j-='0';
                else if ((j >= 'A') && (j <= 'F'))
                        j=j-'A'+10;
                else if ((j >= 'a') && (j <= 'f'))
                        j=j-'a'+10;
                else
                        {
                        BIO_printf(bio_err,"non-hex digit\n");
                        return(0);
                        }
                if (i&1)
                        out[i/2]|=j;
                else
                        out[i/2]=(j<<4);
                }
        return(1);
        }

E.g.,

    char iv_str[] = "12345678901234567890123456789012";
    unsigned char iv[16];
    if( !set_hex(iv_str, iv, sizeof(iv)) )
    {
        // Handle error where string was not a well-formed IV
    }
    printf("IV: "); for(int i=0;i<sizeof(iv); ++i) { printf("%02x", iv[i]); } printf("\n");

Yes! This is exactly what's happening. So how would I go about converting the hex string to the correct bytes? — David Nix, Sep 14 '11 at 16:39
@galacticfury: I've added the function OpenSSL uses. Their implementation is pretty archaic but the algorithm is simple if you want to implement it yourself. For each hex char in your string, convert the char to decimal (0-15). If the hex char is an upper nibble, multiply it by 16. If a lower nibble, add the value to your upper nibble and you have a full byte. — indiv, Sep 14 '11 at 18:15
Unfortunately this routine won't work and it will receive a signal "EXE_BAD_ACCESS". There are five lines of code in the CocoaFu example that convert the key. — zaph, Sep 14 '11 at 19:31
@CocoaFu: The routine works. I fixed my "how to use" example in an edit so it no longer passes an immutable input string (const char *). Because the OpenSSL code clears the input array after converting it so as to not leave plain-text copies of the key and IV lying around memory. So if you were to get an EXC_BAD_ACCESS, for instance, you couldn't run the "strings" utility on your crash dump or core file to get conveniently printable copies of your encryption keys (in theory). The OpenSSL solution is just an implementation of strtol for base 16. — indiv, Sep 14 '11 at 20:05
@indiv: Sorry, it was a tough choice between you and CocoaFu. It's still not decoding, but it is converting to hex properly. CocoaFu's hex conversion method was simpler. I will now do what I should have done in the first place and test incrementally with simple data as CocoaFu suggested. I do appreciate your answer! — David Nix, Sep 15 '11 at 18:08

score 2 · Answer 3 · answered Sep 13 '11 at 22:32

2

To me your 00ff349830193845af43984758690213 looks a lot more like hex digits than Base64. The fact that there are 16 bytes worth of hex makes it a good bet that this is the IV expressed in hex, not in Base64.

answered Sep 13 '11 at 22:32

rossum

15,344
1
24
38

The 00ff349830193845af43984758690213, which is the key, is hex. The iv is also hex. The string I'm trying to decrypt is base 64. – David Nix Sep 14 '11 at 16:40

Unable to decrypt data encoded via OpenSSL on iPhone

3 Answers3

Linked