2

I'm trying to decrypt RC2 data with only 64 effective bits.

Since I can only have 64-bits, it's my understanding that before calling CCCrypt I must use a method to reduce the key down to this many bits. Since I couldn't find any such method in Apple's CommonCrypto library, I am using this keyschedule method I found.

These are the method's parameters:

void rc2_keyschedule( unsigned short xkey[64], 
                     const unsigned char *key, 
                     unsigned len, 
                     unsigned bits ) 

For the actual decryption part, I'm trying to work off of an example that uses AES 256. This is what I have so far:

// setup the key to send to CCCrypt
unsigned char originalKey[16] = /* derived from some other method */;
unsigned short key[64];
unsigned effectiveBits = 64;
rc2_keyschedule(key, originalKey, 16, effectiveBits);
// key is now 128 bytes, and I manually checked it for accuracy

// setup the cipherText to send to CCCrypt
NSData *cipherText = /* derived from some other method */;
// cipherText was manually checked for accuracy

// setup the buffer to send to CCCrypt
size_t bufferSize = [cipherText length] + kCCBlockSizeRC2;
void *buffer = malloc(bufferSize);
size_t numBytesDecrypted = 0;

// call CCCrypt
CCCryptorStatus cryptStatus = CCCrypt(kCCDecrypt, 
                                      kCCAlgorithmRC2, 
                                      kCCOptionPKCS7Padding,
                                      key, 128,
                                      NULL /* initialization vector (optional) */,
                                      [cipherText bytes], [cipherText length],
                                      buffer, bufferSize, /* output */
                                      &numBytesDecrypted);


if (cryptStatus == kCCSuccess) {
  return [NSData dataWithBytesNoCopy:buffer length:numBytesDecrypted];
}

free(buffer); //free the buffer;
return nil;

When I run this, cryptStatus has the value kCCDecodeError which is documented as:

@constant kCCDecodeError Input data did not decode or decrypt properly.

The reason I am sending 128 as the keyLength to CCCrypt is because my key is 64 short ints long, and I believe that 1 short is equal to 2 bytes. Thus, 64 * 2=128.

I have no idea what I need to send for the option. I just used kCCOptionPKCS7Padding which was taken from the AES example. The other options available are kCCOptionECBMode and CBC. When I try the other two options, cryptStatus does become kCCSuccess, but the data is always null. I think that it is erroneously reporting success.

When I say that I "manually checked something for accuracy", I mean that I compared the key and cipher at those points to a JavaScript implementation which works successfully.

How can I use Apple's Libraries (i.e. CommonCrypt) to decrypt RC2 data?

skaffman
  • 398,947
  • 96
  • 818
  • 769
Senseful
  • 86,719
  • 67
  • 308
  • 465
  • 1
    First see if you can get either implementation to agree with the test vectors in RFC 2268. – President James K. Polk Sep 01 '10 at 23:11
  • Seems to me that the rc2_keyschedule mentioned is used *internally* to the RC2 cipher. It would be more logical to try to feed 8 bytes (64 bits) directly to the CCCrypt method, and state this 8 as the key size. It took me about 10 minutes to find out that the key size should be specified in bytes by looking at the constants in CommonCryptor.h (for crying out loud). – Maarten Bodewes Dec 01 '11 at 23:05

0 Answers0