How do I create self signed certificates that two tomcats will be happy with when using https between the tomcats?

Question

I am trying to use https between two tomcat servers. Unfortunately, the self-signed certificates are causing this error:

Caused by: javax.net.ssl.SSLHandshakeException: 
           sun.security.validator.ValidatorException: PKIX path building failed: 
           sun.security.provider.certpath.SunCertPathBuilderException: 
                      unable to find valid certification path to requested target

Specifically, I have a master tomcat and a number of slave tomcat servers. The master communicates from a servlet using a simple HttpURLConnection.

What is the simplest way for me to create self signed certificates using my own self generated Certificate Authority, such that every time I add a new server, I do not need to change the master tomcat server.

I have access to openssl and java 7 keytool

For reference my previous configuration:

The server.xml connector:

<Connector port="443" maxHttpHeaderSize="8192" maxThreads="150" 
    minSpareThreads="25" maxSpareThreads="75" enableLookups="false" 
    disableUploadTimeout="true" acceptCount="100" debug="0" scheme="https" 
    secure="true" clientAuth="false" sslProtocol="TLS" keystoreType="PKCS12" 
    keystoreFile="/usr/java/apache-tomee-plus/conf/keystore.ks" 
    keystorePass="XXX_SSL" truststoreType="JKS" 
    truststoreFile="/usr/java/apache-tomee-plus/conf/truststore.ks" 
    SSLEnabled="true" maxPostSize="0"/>

The startup script /etc/init.d/tomee

$DAEMON_HOME/jsvc \
-user $TOMCAT_USER \
-home $JAVA_HOME \
-pidfile $JSVC_PID_FILE \
-Dcatalina.home=$CATALINA_HOME \
-Djava.security.auth.login.config=$CATALINA_HOME/conf/jaas.conf \
-Djavax.net.ssl.keystore=$CATALINA_HOME/conf/keystore.ks \
-Djavax.net.ssl.keyStorePassword=XXX_SSL \
-Djavax.net.ssl.trustStore=$CATALINA_HOME/conf/truststore.ks \
-Djavax.net.ssl.trustStorePassword=changeit \
-Djava.awt.headless=true \
-Djava.io.tmpdir=$TMP_DIR \
-Dopenam.agents.bootstrap.dir=/home/tomcat/tomcat_v6_agent/Agent_001/config \
-Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses \
-outfile $CATALINA_HOME/logs/catalina.out \
-errfile $CATALINA_HOME/logs/catalina.err \
$CATALINA_OPTS \
-cp $CLASSPATH \
org.apache.catalina.startup.Bootstrap

conf/jaas.conf

josso {
  org.josso.tc55.agent.jaas.SSOGatewayLoginModule required debug=true;
};

Which is there for legacy support only and will be phased out. I'm not sure it even loads since it is built for tomcat 5.5...

Within the code I am avoiding the problems of using IP addresses within the CN= by using the following HostnameVerifier().

HostnameVerifier hv = new HostnameVerifier()
{
    public boolean verify(String urlHostName, SSLSession session)
    {
        return true;
    }
};
HttpsURLConnection.setDefaultHostnameVerifier(hv);

connection = (HttpURLConnection) servlet.openConnection();

------------ Update ---------------

This has been solved by a lengthy discussion with @Bruno, please use his original post and the long chat discussion that we had.

In the end I used the tools Keytool Explorer and XCA to make it easier for me to learn and execute.

Answer 1

Firstly, you're not actually making server-to-server connections: these are still client-to-server connections. It just happens that your client is a webapp running within a servlet container.

It matters because it's the client trust settings that are going to be used to establish the connections. Those are different from the trust settings on the Tomcat connector.

How your client webapp uses its trust settings depends on how it's implemented and on which library it uses. It seems fair to say that most clients would use the default JRE settings, unless there is specific code to do otherwise. Without any settings, the JRE's cacerts file would normally be used (see details in JSSE Reference guide, customisation section). You can also specify the usual javax.net.ssl.* properties (using -Djavax.net.ssl....=....); this would need to be done in the catalina launch script file (.bat or .sh depending on the platform) in JAVA_OPTS. This will affect the default values running within this JRE instance, so it might not be your preferred choice (you might want something more specific that affects only a given webapp, but you'd need to know how that webapp is implemented or which possible options it can use). The settings in your <Connector .../> will override this, though (and the truststore settings there don't matter unless you want to use client-certificate authentication too).

If you have a number of systems, it's probably not worth using self-signed certificates: each client would need to import all these self-signed certificates into its trust store. Instead you can create your own CA and import only that CA certificate. (In a local deployment, a single top-level/root CA should be sufficient, without the need for intermediate CAs.) OpenSSL's CA.pl is a good script to build a small CA, but you may find other tools such as XCA or TinyCA more convenient. (Note that you can generally use .p12 files directly in Java, using the PKCS12 keystore type, instead of the default JKS. For PKCS#12 stores, the keystore password and the key/keymanager password are the same.)

Since you're using IP addresses instead of names, you should also be aware that Java follows RFC 2818 on IP address quite stricly (unlike many browsers). In particular, an IP address in the CN of your Subject DN will not work: it needs to be in a Subject Alternative Name extension (of type IPAddress, not DNSname).

---

EDIT:

First, get rid of that hostname verifier: it introduces a vulnerability to MITM attacks for all the HttpsURLConnection that will use that default verifier, including possible connections to external servers. You can fix this by using an IP address the SAN.

Here are the steps you need to follow:

• Create your CA (e.g. with XCA). Export its certificate (not its private key) to mycacert.pem.

• If your application is likely to make HTTPS connections to other services, copy the default cacerts file into a new file mytruststore.jks (if your application isn't making any other connection, don't copy the file, keytool will create it).

• Import mycacert.pem into mytruststore.jks with keytool:

  keytool -import -keystore mytruststore.jks -alias my_alias_name -cert mycacert.pem
  

• You now have a truststore that you can install in all your clients that will need to talk to these servers. (You might need to perform these steps again if the cacerts file gets updated with the JRE.)

  • Configure that truststore in your clients. Here, this is done in your Catalina scripts in JAVA_OPTS: -Djavax.net.ssl.trustStore=... (and password). Besides the fact these JVM options are in the Catalina launch strict, they're the same as for other types of client.
  • Get rid of the -Djavax.net.ssl.keyStore=... there, this would only be used for client-certificate authentication, and it's not always a good idea to let the webapps use the certificate of the server within which they're running for client authentication.

• Create a certificate with that CA for each server you want to use. Make sure you're using an IP-Address SAN if you want to use IP addresses. (Since you're in control of everything, XCA should allow you to generate the private key and certificate in one step, no need for CSR here, but you could if you wanted to. It also has prepared profiles for web server extensions, as well as the option to add Subject Alternative Names of type IP address or other.)

• Export the combination of certificate and private key into a PKCS#12 (e.g. server1.p12) for each server. Each server should have its own, don't share them. You'll be able to use this as the "keystore keystore" in each server configuration.

  • Configure these keystores in each server in the connector configuration (keystoreType="PKCS12", and keyPass and keystorePass should be identical when using PKCS#12 stores).
  • You don't really need any truststore settings in your connector configuration unless you're using client-certificate authentication.

(It might be worth using client-certificate authentication to make sure the connections to your server at least only come from one other entity coming from that same CA, short of anything more precise, but the truststore you configure on the connector would need to contain only your CA certificate, not the others, so create a brand new truststore for that.)

Answer 2

To answer your question, I don't think that you need a chain of certificates. I already used tomcat with a self signed server certificate and connected to it with another self signed client certificate. I don't know about tomEE but I guess it would be the same.

To help you figure out what is goining on, I think about these possible debugging actions:

• before going into automation, try to make things work without scripting. Generating the right configuration will than be easier.

• make sure the script is running correctly: check if tje contents of the stores are as expected and make sure the server.xml files point to the right ones.

• Try to test the servers separately using a browser client authentication if you can.

Above all, make sure you really need https between your servers. If you are in a protected realm, you are probably over using security and just slowing your application.

Answer 3

How do I create self signed certificates that two tomcats will be happy with when using https between the tomcats?

Tomcat must allow you to do one of two things. First is run a private PKI, or second is utilize a pinset.

---

The easier answer first: a pinset is a set of trusted certificates or public keys to identify a host or set of hosts. It's certificate or public key pinning with a multiplier. In this case, you want to take the 3 or 4 certificates and apply them to each server. Then, when peer's communicate, they will implicitly trust the peer's certificate.

It does not appear Tomcat (or IIS, or Apache) allow you to specify a pinset. That is, you cannot specify a collection of self-signed certificates to trust. So you'll have to use a private PKI.

---

The harder answer is to run a private PKI. There's nothing difficult about running a private PKI. The difficulty is in implementing it because there's a lot to it. There's no silver bullet to make it work because you need the private PKI and you need to configure your servers to use it.

---

For the question, "How do I use my own CA to sign each cert", see for example, How To Setup a CA or even Signing a certificate with my CA and Create my CA how to make a trusted self certified....

---

I can't help you with the configuration of Tomcat. But here are the steps to do it programmatically in OpenSSL if you were building a client and server. Here, "client" and "server" means the role the Tomcat server is taking (each takes both roles):

• Client

  • call SSL_CTX_load_verify_locations with your CA
  • call SSL_CTX_set_default_verify_paths on the context
  • call SSL_CTX_set_verify with SSL_VERIFY_PEER

• Server

  • call SSL_CTX_use_certificate_chain_file
  • call SSL_CTX_use_PrivateKey_file
  • call SSL_CTX_set_verify with SSL_VERIFY_PEER|VERIFY_FAIL_IF_NO_PEER_CERT

When acting as a client, the Tomcat server needs to know the organization's trusted root. That's what SSL_CTX_load_verify_locations and SSL_CTX_set_default_verify_paths accomplish.

When acting as a client, SSL_CTX_set_verify ensures the Tomcat server verifies the peer's identify (modulo the lack of hostname checking in OpenSSL).

When acting as a server, the Tomcat server needs to know the certificate to send and the private key to sign with. That's what SSL_CTX_use_certificate_chain_file and SSL_CTX_use_PrivateKey_file do.

When acting as a server, the the Tomcat server wants mutual authentication. That's what SSL_CTX_set_verify with SSL_VERIFY_PEER|VERIFY_FAIL_IF_NO_PEER_CERT does.

I don't know how to pound these into an Tomcat configuration, though.