Tomcat : Installing LetsEncrypt certificate for https, not working

Question

I have an instance of Apache Tomcat running on our server, for which I am trying to use LetsEncrypt for https. I found some scripts to import the certificate, and I can check in the keystore that the certificate is imported.

Unfortunately, https is not working. Whenever I try to connect, it just keeps waiting(progress bar rotating). I don't know what I am doing wrong.

Script :

cd /var/git/letsencrypt
#git pull origin master
iptables -I INPUT -p tcp -m tcp --dport 9999 -j ACCEPT
iptables -t nat -I PREROUTING -i $networkdevice -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9999

./letsencrypt-auto certonly --standalone --test-cert --break-my-certs -d $mydomain --renew-by-default --email $myemail --agree-tos


iptables -t nat -D PREROUTING -i $networkdevice -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9999
iptables -D INPUT -p tcp -m tcp --dport 9999 -j ACCEPT

$keytooldir/keytool -delete -alias root -storepass NEWPASS -keystore $keystoredir
$keytooldir/keytool -delete -alias tomcat -storepass NEWPASS -keystore $keystoredir

openssl pkcs12 -export -in $certdir/fullchain.pem -inkey $certdir/privkey.pem -out $certdir/cert_and_key.p12 -name tomcat -CAfile $certdir/chain.pem -caname root -password pass:aaa

$keytooldir/keytool -importkeystore -srcstorepass aaa -deststorepass NEWPASS -destkeypass changeit -srckeystore $certdir/cert_and_key.p12 -srcstoretype PKCS12 -alias tomcat -keystore $keystoredir
$keytooldir/keytool -import -trustcacerts -alias root -deststorepass NEWPASS -file $certdir/chain.pem -noprompt -keystore $keystoredir

Output :

iptables v1.4.21: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.
Checking for new version...
Requesting root privileges to run letsencrypt...

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at
   /etc/letsencrypt/live/domina-name/fullchain.pem.
   Your cert will expire on 2016-07-27. To obtain a new version of the
   certificate in the future, simply run Let's Encrypt again.
iptables v1.4.21: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.
Certificate was added to keystore

Apache tomcat connector :

 <Connector port="443" protocol="org.apache.coyote.http11.Http11NioProtocol" SSLEnabled="true" maxThreads="200"
              scheme="https" secure="true" clientAuth="false"  sslProtocol="TLS"
                keystoreFile="/root/.keystore" keystorePass="newPASS" />

OpenSSL connection output :

openssl s_client -connect domain_name.de:443
CONNECTED(00000003)

It is stuck at connected.

What am I doing wrong. Any ideas? Thank you.

Update

As I checked, I was getting a cannot recover key exception. I thought it might be because Tomcat requires Key-pass, so I changed the password of the key with the following command, but still error continues :

openssl rsa -aes192 -in yourprivatekeywithoutpassword.pem -out newprivatekeywithpassword.pem

Error log :

SEVERE: Failed to initialize end point associated with ProtocolHandler ["http-nio-443"]
java.security.UnrecoverableKeyException: Cannot recover key
        at sun.security.provider.KeyProtector.recover(KeyProtector.java:328)
        at sun.security.provider.JavaKeyStore.engineGetKey(JavaKeyStore.java:146)
        at sun.security.provider.JavaKeyStore$JKS.engineGetKey(JavaKeyStore.java:56)
        at sun.security.provider.KeyStoreDelegator.engineGetKey(KeyStoreDelegator.java:96)
        at sun.security.provider.JavaKeyStore$DualFormatJKS.engineGetKey(JavaKeyStore.java:70)
        at java.security.KeyStore.getKey(KeyStore.java:1023)
        at sun.security.ssl.SunX509KeyManagerImpl.<init>(SunX509KeyManagerImpl.java:133)
        at sun.security.ssl.KeyManagerFactoryImpl$SunX509.engineInit(KeyManagerFactoryImpl.java:70)
        at javax.net.ssl.KeyManagerFactory.init(KeyManagerFactory.java:256)

Answer 1

I was finally able to solve this problem by changing the server URL from which the certificate is retrieved. AS you can see I was getting an error that the certificate is not trusted. This was happening because the certificate was from a staging server.

I changed the configuration file with server parameter in renewal to

https://acme-v01.api.letsencrypt.org/directory

After that I retrieved all the certificates from letsEncrypt, and installed them again.

Output :

openssl s_client -connect domain.de:443
CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = domain.de
verify return:1
---
  Verify return code: 0 (ok)

This problem is hereby solved and finished. Thank you for your time.

Answer 2

The only thing that I can think of is that you have a different keystorePass and keyPass without providing keyPass on the connector. Tomcat tries to access the key with the same password as the keystore, fails, and produces the error you see.

In your update, you updated the pem file; did you import it back into the keystore to update what tomcat is using? I assume that you did, but have to ask for completeness. If you didn't, then the key password change does nothing to the keystore that tomcat is using.

The only solution that I can offer is adding the keyPass attribute to your connector to see if that works. I tested locally and having different keystore and key passwords does produce the error. Adding keyPass to the connector solves the issue for me.

Answer 3

For your problem, I want to modulerize your problems.

At first,

iptables v1.4.21: can't initialize iptables table `nat': Table does not exist (do you need to insmod?) Perhaps iptables or your kernel needs to be upgraded.

For this problem, it can be solved by changing ip_tables location. Command will be look like below:

sudo modprobe ip_tables
sudo echo 'ip_tables' >> /etc/modules

For full section, you can go through this link: iptables: Table does not exist (do you need to insmod?)

Resource Link:

1. iptables v1.4.14: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
2. iptables nat does not exist

---

Secondly, I want to fix the section

java.security.UnrecoverableKeyException: Cannot recover key

This problem actually happens when the keymanager use wrong/bad password which is actually mismatch or sometimes uppercase lowercase issue or your keystore may be corrupted. You must ensure that KeyStore password and The Keypassword are same.

For this you can check by 2 ways-

You can change/list the key in keytool with the same password or another

keytool -keypasswd -new changeit -keystore cacerts -storepass changeit -alias someapp -keypass password

The above example changes the password from password to changeit. This command will succeed if the private key password was password.

You can also use alias. You'll need to implement getPrivateKey(String alias) in your custom X509KeyManager to take this into account. In particular, it will have to load the keys from your KeyStore instance with the right password for each alias (see getKey(String alias, char[] password)).

Credit goes to @bruno

---

Next,

Another thing I have seen that, you have used sometimes NEWPASS in command mode and newPASS in tomcat connector as password. Hope there is some mismatch. Please check and verify.

Tomcat Recommendation for preparing the certificate keystore:

The default password used by Tomcat is "changeit" (all lower case), although you can specify a custom password if you like. You will also need to specify the custom password in the server.xml configuration file

You MUST use the same password here as was used for the keystore password itself. This is a restriction of the Tomcat implementation.

The port attribute is the TCP/IP port number on which Tomcat will listen for secure connections. You can change this to any port number you wish (such as to the default port for https communications, which is 443). However, special setup (outside the scope of this document) is necessary to run Tomcat on port numbers lower than 1024 on many operating systems.

After completing configuration changes, you must restart Tomcat

You can also go through SSL Support

Resource Link:

1. Can not get key from KeyStore

---

UPDATE:

If first portion not works properly then follow as @douglas's service provider answered like below:

This is a limitation of the virtualization system we use (OpenVZ), basic iptables rules are possible but not those who use the nat table.

If this really is a problem, we can offer you to migrate to a other system virtualization (KVM) as we begin to offer our customers.

SO you had to migrate your server to the new system. If possible you can try it.