3DES key exchange with RSA in Java

Question

I'm implementing a WebService in Java in which the server needs to send a 3DES key to the client using the RSA algorithm. The symmetric is generated by the server. Both the server and the client have their own RSA key-pairs, that were previously exchanged.

In this code, the server sends the symmetric key to the client.

@WebMethod
public byte[] getSymmetricKey(){
    try{
        Cipher cipher = Cipher.getInstance("RSA");

        // First, encrypts the symmetric key with the client's public key
        cipher.init(Cipher.ENCRYPT_MODE, this.clientKey);
        byte[] partialCipher = cipher.doFinal(this.key.getBytes());

        // Finally, encrypts the previous result with the server's private key
        cipher.init(Cipher.ENCRYPT_MODE, this.privateKey);
        byte[] cipherData = cipher.doFinal(partialCipher);

        return cipherData;
    }catch (Exception ex){
        ex.printStackTrace();
    }

}

When I run the encryption with the server's private key, I get an error of IllegalBlockSizeException. Why do I get this exception if the padding is activated by default? I've also tried explicitly activate the padding with Cipher.getInstance("RSA/ECB/PKCS1Padding"). Finally, here's the exception output:

    SEVERE: javax.crypto.IllegalBlockSizeException: Data must not be longer than 245 bytes
javax.crypto.IllegalBlockSizeException: Data must not be longer than 245 bytes
    at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:346)
    at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:391)
    at javax.crypto.Cipher.doFinal(Cipher.java:2087)
    at server.FileTransfererImpl.getSymmetricKey(FileTransfererImpl.java:112)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at sun.reflect.misc.Trampoline.invoke(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at sun.reflect.misc.MethodUtil.invoke(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at com.sun.xml.internal.ws.api.server.MethodUtil.invoke(Unknown Source)
    at com.sun.xml.internal.ws.api.server.InstanceResolver$1.invoke(Unknown Source)
    at com.sun.xml.internal.ws.server.InvokerTube$2.invoke(Unknown Source)
    at com.sun.xml.internal.ws.server.sei.EndpointMethodHandler.invoke(Unknown Source)
    at com.sun.xml.internal.ws.server.sei.SEIInvokerTube.processRequest(Unknown Source)
    at com.sun.xml.internal.ws.api.pipe.Fiber.__doRun(Unknown Source)
    at com.sun.xml.internal.ws.api.pipe.Fiber._doRun(Unknown Source)
    at com.sun.xml.internal.ws.api.pipe.Fiber.doRun(Unknown Source)
    at com.sun.xml.internal.ws.api.pipe.Fiber.runSync(Unknown Source)
    at com.sun.xml.internal.ws.server.WSEndpointImpl$2.process(Unknown Source)
    at com.sun.xml.internal.ws.transport.http.HttpAdapter$HttpToolkit.handle(Unknown Source)
    at com.sun.xml.internal.ws.transport.http.HttpAdapter.handle(Unknown Source)
    at com.sun.xml.internal.ws.transport.http.server.WSHttpHandler.handleExchange(Unknown Source)
    at com.sun.xml.internal.ws.transport.http.server.WSHttpHandler.handle(Unknown Source)
    at com.sun.net.httpserver.Filter$Chain.doFilter(Unknown Source)
    at sun.net.httpserver.AuthFilter.doFilter(Unknown Source)
    at com.sun.net.httpserver.Filter$Chain.doFilter(Unknown Source)
    at sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(Unknown Source)
    at com.sun.net.httpserver.Filter$Chain.doFilter(Unknown Source)
    at sun.net.httpserver.ServerImpl$Exchange.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

On what line does this exception occurr? You shouldn't re-use `Cipher` instances. Also, you cannot encrypt with an RSA private key. — Artjom B., Dec 05 '14 at 18:22
@MartinKonecny Yes, it is. The thing is that it shows no exception when I'm using ONLY the server's private key for this task. It only throws an exception when I try to use two different keys (server's private and client's public). The partialCipher (which the data for the second step) isn't too large (256 bytes), but maybe the RSA algorithm is meant to encrypt really small data only? — Vítor Antero, Dec 05 '14 at 18:28
@ArtjomB. Hum, I didn't know about that. Strange thing is, that I'm able to encrypt using only the server's private key. Maybe I should implement a proper Key Exchange Protocol and generate different 3DES keys in both sides (server and client) ? Thank you. — Vítor Antero, Dec 05 '14 at 18:30
"Encrypting" with the RSA private key means signing and it doesn't work the same was as actual encryption with an RSA public key. I'm not that versed in the Java intricacies of Signature generation t provide you an answer. You would usually use something like in this question http://stackoverflow.com/questions/521101/using-sha1-and-rsa-with-java-security-signature-vs-messagedigest-and-cipher. Note that SHA1 is not safe anymore. A proper Diffie-Hellman Key Exchange would be more appropriate if you want (perfect) forward secrecy. — Artjom B., Dec 05 '14 at 18:36
It's possible to perform RSA encryption of a symmetric key and send it to another party. In your case you could simply encrypt with the public key of the client and then decrypt using almost the same code as you have now (this is also how the RSA_ suites of SSL work, although there the client generates the key and encrypts it with the public key of the server). Never ever try and encrypt with a private key - it's most likely not the same as signing either [because of the padding and other differences](http://crypto.stackexchange.com/q/15997/1172). You don't get forward secrecy this way though. — Maarten Bodewes, Dec 06 '14 at 00:00
PS why the hack do you use 3DES? It's the end of 2014 already. — Maarten Bodewes, Dec 06 '14 at 00:41

score 1 · Accepted Answer · answered Sep 21 '16 at 22:49

I was researching soem stuff today and found this question. Since it has not been answered, I'll leave this here for future reference.

According with PKCS #1, the RSAES-PKCS1-V1_5-ENCRYPT algorithm can encrypt up to k - 11 bytes where k is the "size" of the key in bytes. Those 11 bytes are used for "headers".

If you are using a 2048 bits RSA key, that gives you k = 256 and you can encrypt up to 256 - 11 = 245 bytes of data.

Check the actual size of this.key.

3DES key exchange with RSA in Java

1 Answers1