I am currently working on a project that involves mutual SSL between a client and a server. I was able to do this using P12 and JKS files but am now required to do the same thing using a smartcard token.
First I was quite confused that I could not obtain the private key from the card (we're talking unextractable keys here), but now I think I understand the basics of the whole process (correct me if I'm wrong, please):
Instead of reading the private key directly, I let the smartcard handle the key via the provider (we use OnePin OpenSC PKCS11 in Linux and SunMSCAPI in Windows). This way, the key is never actually extracted from the card.
As the client we're talking about is quite a complex thing, we decided to create a simpler test project that would first try to use the smartcard's token for basic cryptographic operations like signing and ciphering.
What I have successfully done
I managed to get signing to work in Linux with the following code:
Provider sunMSC = Security.getProvider("SunMSCAPI");
final String alias = "Some Alias of a KeyEntry";
final byte[] msg = "Hello World".getBytes();
KeyStore.CallbackHandlerProtection chp = new KeyStore.CallbackHandlerProtection(new FixedPINCallbackHandler("12345"));
KeyStore.Builder builder = KeyStore.Builder.newInstance("Windows-MY", sunMSC, chp);
KeyStore store = builder.getKeyStore();
PrivateKey pk = (PrivateKey)store.getKey(alias, null);
Signature signature = Signature.getInstance("SHA1withRSA", sunMSC);
signature.initSign(pk);
signature.update(msg);
final byte[] sig = signature.sign();
I can verify that signature using the corresponding certificate. The FixedPINCallbackHandler class is very simple and just sets the password of each PasswordCallback that it encounters in its handle-Method with a given PIN (12345 in the example's case). I believe that the PIN does not really matter, as both Linux and Windows ask for the PIN using the reader's drivers when the above code is executed (I tried both the valid and an invalid PIN, giving me the same results. Windows will complain if you do not set any key in the callback handling code, though).
What the actual problem is
The problem is that the above code that works totally fine in Linux won't work in WIndows. The moment it hits the last line in the snippet above (when signature.sign() is called), it would throw the following exception:
Exception in thread "main" java.security.SignatureException: Ungültiger Schlüssel
at sun.security.mscapi.RSASignature.signHash(Native Method)
at sun.security.mscapi.RSASignature.engineSign(RSASignature.java:390)
at java.security.Signature$Delegate.engineSign(Unknown Source)
at java.security.Signature.sign(Unknown Source)
at (my code).
(I'm from Germany, the "Ungültiger Schlüssel" from the above stack trace means "invalid key").
The class of the PrivateKey is sun.security.mscapi.RSAPrivateKey.
When I tried to establish a connection to the server earlier, I would always get something like this:
*** CertificateVerify
DAV-WS-Mirror, WRITE: TLSv1 Handshake, length = 134
DAV-WS-Mirror, WRITE: TLSv1 Change Cipher Spec, length = 1
*** Finished
verify_data: { 122, 176, 195, 78, 166, 117, 103, 191, 102, 9, 101, 255 }
***
DAV-WS-Mirror, WRITE: TLSv1 Handshake, length = 40
DAV-WS-Mirror, received EOFException: error
DAV-WS-Mirror, handling exception: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
DAV-WS-Mirror, SEND TLSv1 ALERT: fatal, description = handshake_failure
DAV-WS-Mirror, WRITE: TLSv1 Alert, length = 24
DAV-WS-Mirror, called closeSocket()
[DEBUG] [2013-12-04 15:11:40,809] [HTTPSender-146] - [javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake]
(DAV-WS-Mirror is just the name of a thread, should anyone wonder).
Private key details
Just in case it's important, here's what pkcs15-tool -k tells me about the private key on the smartcard:
Private RSA Key [Private Key]
Object Flags : [0x3], private, modifiable
Usage : [0x4], sign
Access Flags : [0x1D], sensitive, alwaysSensitive, neverExtract, local
ModLength : 2048
Key ref : 1 (0x1)
Native : yes
Path : 3f005015
Auth ID : 01
ID : 45
What I tried
I tried a second smartcard (which is actually the type of card we need to support, which is different from the one I own), but got the same stacktrace.
I tried installing the unlimited strength policy files (JCE), but the stacktrace stayed the same (it's possible that they were installed already).
Different versions of Java. The final software must work with Java 6, but I wanted to rule out any problems with the different versions (yes, the client runs on the 32-Bit version).
What I want to know/do
I'd really love to make this code work in both Windows and Linux (as the client software targets mostly Windows machines, the windows part is more important) and later on be able to create a mutually authenticated SSL connection between the client and the server using the smartcards.
I am quite free in choosing libraries, so I have absolutely no problem with moving away from SunMSCAPI if there's an alternative.
Should you have any questions left, please feel free to ask, I will answer them as fast and thoroughly as I can.
Any help is highly appreciated, thank you guys in advance.
EDIT
I finally managed to sign data in Windows by simply installing OpenSC and using their DLL instead of the MSCAPI (after I found this article on SO).
The code remains the same, but instead of Security.getProvider("SunMSCAPI"), it now uses new sun.security.pkcs11.SunPKCS11(new FileInputStream(new File(configFile))) where configFile is simply the fully qualified path of a file containing the following:
name=OpenSC
library=C:\\opensc-pkcs11.dll
slot=1
Using this code, I can sign data, as I mentioned before.
I then tried to create the mutually authenticated SSL connection to my server software. The last thing I see from the SSL Debug log is (snip):
Server write IV:
0000: 79 85 7B D7 D0 5C 53 E6 5E CE 49 EA DF 94 BD 1D y....\S.^.I.....
DAV-WS-Mirror, SEND TLSv1 ALERT: fatal, description = handshake_failure
DAV-WS-Mirror, WRITE: TLSv1 Alert, length = 2
DAV-WS-Mirror, called closeSocket()
DAV-WS-Mirror, handling exception: javax.net.ssl.SSLHandshakeException: Error signing certificate verify
DAV-WS-Mirror, IOException in getSession(): javax.net.ssl.SSLHandshakeException: Error signing certificate verify
I am guessing that the client is required to verify that it owns the private key related to the public certificate it presented to the server during the handshake, but that the Provider (OpenSC) fails to sign the challenge (again, I'm going for an educated guess here).
We tested the exact same smartcard using the exact same card reader using Firefox and were able to complete the handshake.
Does anyone have any idea how this could be done?
