Data encrypted in C# is 1 byte too long to be decrypted in Java

Question

I have a server written in Java which sends converts its RSA key to the XML format used by .NET before sending it to the client:

public String getPublicKeyXML() {
    try {
        KeyFactory factory = KeyFactory.getInstance("RSA");
        RSAPublicKeySpec publicKey = factory.getKeySpec(this.keyPair.getPublic(), RSAPublicKeySpec.class);

        byte[] modulus = publicKey.getModulus().toByteArray();
        byte[] exponent = publicKey.getPublicExponent().toByteArray();

        String modulusStr = Base64.encodeBytes(modulus);
        String exponentStr = Base64.encodeBytes(exponent);

        String format = 
            "<RSAKeyValue>" +
                "<Modulus>%s</Modulus>" +
                "<Exponent>%s</Exponent>" +
            "</RSAKeyValue>";

        return String.format(format, modulusStr, exponentStr);
    } catch (Exception e) {
        this.server.logException(e);
        return "";
    }
}

The client, written in C#, then loads the key and uses it to encrypt a 256 bit AES key:

    public static byte[] encrypt(string xmlKey, byte[] bytes)
    {
        RSACryptoServiceProvider rsa = new RSACryptoServiceProvider();
        rsa.FromXmlString(xmlKey);
        byte[] cipherBytes = rsa.Encrypt(bytes, false);
        rsa.Clear();
        return cipherBytes;
    }

The server is then supposed to decrypt the AES key using its private RSA key:

public byte[] decrypt(byte[] data) {
    try {
        PrivateKey privateKey = this.keyPair.getPrivate();
        Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");
        cipher.init(Cipher.DECRYPT_MODE, privateKey);
        byte[] cipherData = cipher.doFinal(data);
        return cipherData;
    } catch (Exception e) {
        this.server.logException(e);
        return new byte[0];
    }
}

However, the server fails with an error stating "Data must not be longer than 384 bytes." Looking at the data to be decrypted, I noticed that it's 385 bytes. I tried increasing the RSA key length, and now the server tells me the data must be no longer than 512 bytes, while the encrypted data from the client is 513 bytes. Why is the encrypted data always one byte longer than expected?

EDIT:

Here's a sample XML-formatted key as is transmitted from the server to the client:

<RSAKeyValue><Modulus>ANsMd2dCF6RsD5v5qjlHEjHm0VWD99gSYHP+pvyU8OgNL9xM5+o+yMAxWISOwMii9vJk1IzYGf18Fj2sMb5BsInlG2boZHb6KHh7v8ObPa4MuwB/U63i8AVU3N/JTugaPH0TKvo1WNUooXEHT23nOk+vh1QipzgKQYGl68qU35vKmpNAa79l1spXA66LckTWal4art9T08Rxgn9cMWujlF+wh9EQKQoxxgj4gCoXWRDTFYjRo/Mp5xDPwNjloTs/vFCPLvY7oI+lVrHhrPyz1R473ZuEhZm+rSeGBcY9I8vhg0AIixN7KYBLhrIecmqoNZHi6LohjD2F9zhdLaTU0IIU8eeKpbEZ5eB1kYngMONBq3A/IoG0Qa/7EcSAMMspBEObffK9kCNzvnbFg5wLuy8EHNaK3nmnuTppgCwCyNqZyHeAbZaUBjNguLhHtqkHFiPJ063Xesj9UbSsCmlBliGTDXWfeJANnjGP6D3R+uLXVy9SZe+cY92JW3eZA2k//w==</Modulus><Exponent>AQAB</Exponent></RSAKeyValue>

I have verified that the data sent is the same as the data being received.

Knocking off the last byte results in a BadPaddingException. I also tried knocking off the first byte, with the same result.

Answer 1

I found the problem. The BigInteger's toByteArray() function included a leading zero for some reason. I just removed the leading zeros from the array and it now works like a charm!

Answer 2

This will not fix the problem (I tested it to no avail), but I wanted to call to your attention that RSACryptoServiceProvider implements the IDisposable interface and therefore should be properly disposed of when complete. Your C# encrypt method can be written a bit better (and more concise!) as such:

    public static byte[] encrypt(string xmlKey, byte[] bytes)
    {
        using (var rsa = new RSACryptoServiceProvider())
        {
            rsa.FromXmlString(xmlKey);
            return rsa.Encrypt(bytes, false);
        }
    }