8

FINAL EDIT: SOLVED, upgrading local dev to railo 3.3.4.003 resolved the issue.

I have to RC4 encrypt some strings and have them base64 encoded and I'm running into a situation where the same input will generate different outputs on 2 different dev setups.

For instance, if I have a string test2@mail.com
On one machine (DEV-1) I'll get: DunU+ucIPz/Z7Ar+HTw=
and on the other (DEV-2) it'll be: DunU+ucIlZfZ7Ar+HTw=

First, I'm rc4 encrypting it through a function found here. Next I'm feeding it to: toBase64( my_rc4_encrypted_data, "iso-8859-1")

As far as I can tell the rc4 encryption output is the same on both (or I'm missing something). Below are SERVER variables from both machines as well as the encryption function.

Is this something we'll simply have to live with or is there something I can do to 'handle it properly' (for a lack of a better word). I'm concerned that in the future this will bite me and wonder it it can be averted.

edit 1: Output from my_rc4_encrypted_data.getBytes() returns: dev-1:

Native Array (byte[])
14--23--44--6--25-8-63-63--39--20-10--2-29-60

dev-2:

Native Array (byte[])
14--23--44--6--25-8-63-63--39--20-10--2-29-60

(no encoding passed to getBytes() )

DEV-1 (remote)

server.coldfusion
productname Railo
productversion  9,0,0,1

server.java
archModel   64
vendor  Sun Microsystems Inc.
version 1.6.0_26

server.os
arch    amd64
archModel   64
name    Windows Server 2008 R2
version 6.1

server.railo
version 3.3.2.002

server.servlet
name    Resin/4.0.18

DEV-2 (local)

server.coldfusion
productname     Railo
productversion  9,0,0,1

server.java
vendor  Oracle Corporation
version 1.7.0_01

server.os
arch    x86 
name    Windows 7
version 6.1

server.railo
version 3.2.2.000

server.servlet
name    Resin/4.0.18

RC4 function:

function RC4(strPwd,plaintxt) {
  var sbox = ArrayNew(1);
  var key = ArrayNew(1);
  var tempSwap = 0;
  var a = 0;
  var b = 0;
  var intLength = len(strPwd);
  var temp = 0;
  var i = 0;
  var j = 0;
  var k = 0;
  var cipherby = 0;
  var cipher = "";

  for(a=0; a lte 255; a=a+1) {  
    key[a + 1] = asc(mid(strPwd,(a MOD intLength)+1,1));
    sbox[a + 1] = a;
  }

  for(a=0; a lte 255; a=a+1) {  
    b = (b + sbox[a + 1] + key[a + 1]) Mod 256;   
    tempSwap = sbox[a + 1];
    sbox[a + 1] = sbox[b + 1];
    sbox[b + 1] = tempSwap;    
  }

  for(a=1; a lte len(plaintxt); a=a+1) {  
    i = (i + 1) mod 256;
    j = (j + sbox[i + 1]) Mod 256;    
    temp = sbox[i + 1];
    sbox[i + 1] = sbox[j + 1];
    sbox[j + 1] = temp;
    k = sbox[((sbox[i + 1] + sbox[j + 1]) mod 256) + 1];    
    cipherby = BitXor(asc(mid(plaintxt, a, 1)), k);
    cipher = cipher & chr(cipherby);      
  }
  return cipher;
}
Jeromy French
  • 11,812
  • 19
  • 76
  • 129
vector
  • 7,334
  • 8
  • 52
  • 80
  • 1
    Just for the sake of testing, can you switch them both to run on the same JVM version, and see if that makes a difference. It's an easy variable to eliminate. Next would be to try on same version of Railo, in case there's a diff there. – Adam Cameron Jan 11 '13 at 22:43
  • Seems odd. The RC4 function should provide the same output given the same input. I would try Base64 encoding something simple on both machines. I see you have different architectures and Base64 encodes the binary representation into a string so I don't know if that might cause a difference – Stuart Wakefield Jan 11 '13 at 22:50
  • Makes sense, but that's out of my 'jurisdiction' unfortunately. The best I'll be able to do is kick up the 'chain of command'. – vector Jan 11 '13 at 22:51
  • @Stuart Wakefield: rc4 indeed seems to be return same result on both systems, the discrepancy is in the toBase64() – vector Jan 11 '13 at 22:52
  • Do you get different values if you plug `my_rc4_encrypted_data` into `ToBinary()`. Scratch that I mean `my_rc4_encrypted_data.getBytes()` – Stuart Wakefield Jan 11 '13 at 22:56
  • 2
    Good idea. But be sure to use the same encoding in your test ie `String.getBytes(encoding)` (Edit) If you omit it, the jvm default is used. That may be different on your two machines. Hm.. that might be related. On second thought, try it both ways. – Leigh Jan 11 '13 at 23:00
  • 4
    Leigh's probably right - [RAILO-1393](https://issues.jboss.org/browse/RAILO-1393) resulted in a [change](https://github.com/getrailo/railo/commit/7ded821318ba9919f1a899929478fdbd7bf5d0d2#railo-java/railo-core/src/railo/runtime/functions/string/ToBase64.java) to toBase64 related to charset encodings in 3.3.0.017, which is between the 3.3.2.002 and 3.2.2.000 versions you are using. – Peter Boughton Jan 11 '13 at 23:03
  • 2
    Good one Peter. A `charset` difference would explain it. – Leigh Jan 11 '13 at 23:07
  • Yeah that would do it. Good find Peter – Stuart Wakefield Jan 11 '13 at 23:11
  • ... hint on encoding for getBytes(), straight default seems to return the same array from both. – vector Jan 11 '13 at 23:17
  • 1
    Yeah it seems to be the way a passed charset is handled has changed between the versions. If you pass "iso-8859-1" as a parameter you should get two different results. I think the recommendation is to upgrade Railo if you can. – Stuart Wakefield Jan 11 '13 at 23:22
  • ... hm, that might be doable. Thanks. – vector Jan 11 '13 at 23:25
  • @Peter Boughton, if you like to turn your comments into an answer, I'll accept it. – vector Feb 01 '13 at 14:52

2 Answers2

2

Leigh wrote:

But be sure to use the same encoding in your test ie String.getBytes(encoding) (Edit) If you omit it, the jvm default is used.

Leigh is right - RAILO-1393 resulted in a change to toBase64 related to charset encodings in 3.3.0.017, which is between the 3.3.2.002 and 3.2.2.000 versions you are using.

Peter Boughton
  • 110,170
  • 32
  • 120
  • 176
0

As far as I can tell the rc4 encryption output is the same on both (or I'm missing something). Below are SERVER variables from both machines as well as the encryption function.

I would suggest saving the output to two files and then comparing the file size or, even better, a file comparison tool. Base64 encoding is a standard approach to converting binary data into string data.

Assuming that your binary files are both exactly 100% the same, on both of your servers try converting the data to base 64 and then back to binary again. I would predict that only one (or neither) of the servers are able to convert the data back to binary again. At that point, you should have a clue about which server is causing your problem and can dig in further.

If they both can reverse the base 64 data to binary and the binary is correct on both servers... well, I'm not sure.

Doug Hughes
  • 931
  • 1
  • 9
  • 21