Java - SSLServerSocket with only TLS

Question

I am trying to open an SSLServerSocket with custom keystore/truststore and with only TLSv1.2 enabled. Here is my related code for opening such socket:

SSLContext sslContext = null;
ServerSocket serverSocket = null;
KeyManagerFactory kmf = null;
KeyStore keystore = loadKeyStore(KEYSTORE_FILE);
if (keystore == null) {
    // throw exception
}
char[] psw = System.console().readPassword("Enter password for the key materials in file \"%s\":", KEYSTORE_FILE);
try {
    kmf = KeyManagerFactory.getInstance("PKIX");
    kmf.init(keystore, psw);
} catch (NoSuchAlgorithmException | UnrecoverableKeyException | KeyStoreException e) {
    e.printStackTrace();
    kmf = null;
    // throw exception
}
try {
    sslContext = SSLContext.getInstance("TLSv1.2");
    System.out.println(kmf==null); // prints false
    sslContext.init(kmf==null?null:kmf.getKeyManagers(), null, null);
} catch (NoSuchAlgorithmException | KeyManagementException e) {
    // throw exception
}

try {
    serverSocket = sslContext.getServerSocketFactory().createServerSocket(PORT, BACKLOG, HOST);
    ((SSLServerSocket)serverSocket).setEnabledProtocols(new String[]{"TLSv1.2"});
} catch (IOException e) {
    // throw exception
}

the loadKeyStore function is,

private static KeyStore loadKeyStore(String filename) {
    KeyStore keystore = null;
    FileInputStream fis = null;
    try {
        keystore = KeyStore.getInstance("JKS");
        char[] psw = System.console().readPassword("Enter password for the KeyStore file \"%s\":", filename);
        if (psw != null) {
            fis = new FileInputStream(filename);
            keystore.load(fis, psw);
        }
    } catch (KeyStoreException | NoSuchAlgorithmException | CertificateException | IOException e) {
        keystore = null;
        LogManager.getLogger().fatal("cannot load KeyStore from file \"" + filename + "\".", e);
    } finally {
        if (fis != null) {
            try {
                fis.close();
            } catch (IOException e) {
                LogManager.getLogger().error("cannot close file " + filename, e);
            }
            fis = null;
        }
    }
    return keystore;
}

I accept connections in a different thread as

while (!stopped) {
    Socket socket = null;
    try {
        socket = serverSocket.accept();
    } catch (IOException e) {
        if (!stopped) {
            logger.error("exception while accepting connections.", e);
        }
        break;
    }
    // start new threads to handle this connection
}

The problem is, when I enter https://HOST:PORT at Firefox, it says:

Firefox cannot guarantee the safety of your data on HOST because it uses SSLv3, a broken security protocol. Advanced info: ssl_error_no_cypher_overlap

How can I open a server socket that accepts only TLSv1.2 connections?

P.S. I have tried changing "TLSv1.2" strings in the code to "TLS", one by one, but nothing changed.

EDIT: I edited the code as follows:

serverSocket = sslContext.getServerSocketFactory().createServerSocket(port, backlog, host);
((SSLServerSocket)serverSocket).setEnabledProtocols(new String[]{"TLSv1.2"});
for (String s: ((SSLServerSocket)serverSocket).getEnabledCipherSuites()) {
    System.out.println(s);
}

and the output is,

TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 TLS_RSA_WITH_AES_128_CBC_SHA256 TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA TLS_RSA_WITH_AES_128_CBC_SHA TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA TLS_ECDH_RSA_WITH_AES_128_CBC_SHA TLS_DHE_RSA_WITH_AES_128_CBC_SHA TLS_DHE_DSS_WITH_AES_128_CBC_SHA TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256 TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256 TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 TLS_DHE_DSS_WITH_AES_128_GCM_SHA256 TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA SSL_RSA_WITH_3DES_EDE_CBC_SHA TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA TLS_ECDHE_ECDSA_WITH_RC4_128_SHA TLS_ECDHE_RSA_WITH_RC4_128_SHA SSL_RSA_WITH_RC4_128_SHA TLS_ECDH_ECDSA_WITH_RC4_128_SHA TLS_ECDH_RSA_WITH_RC4_128_SHA SSL_RSA_WITH_RC4_128_MD5 TLS_EMPTY_RENEGOTIATION_INFO_SCSV

I am not sure, but it seems the problem is not about missing enabled cipher suites. Right?

EDIT2: I have tried openssl s_client -connect HOST:PORT, and the result is

Answer 1

(There's a very similar question which I've answered here.)

Essentially, SSLContext.getInstance("TLSv1.2") can return an instance that supports other protocols too.

If you want to use a specific set of protocols, you need to use setEnabledProtocols(...), which is what you've done following your first edit. You now get some cipher suites with a name starting with SSL_, but that's just the name, these are still valid for TLS 1.2. As the Java Cryptography Architecture Standard Algorithm Name Documentation for JDK 8 says:

Some JSSE cipher suite names were defined before TLSv1.0 was finalized, and were therefore given the SSL_ prefix. The names mentioned in the TLS RFCs prefixed with TLS_ are functionally equivalent to the JSSE cipher suites prefixed with SSL_.

Your last problem ("no peer certificate available", along with the handshake failure) seems to suggest there is no certificate (with its private key) found in the keystore you're trying to use.

Indeed, although the cipher suites you're mentioning are enabled, they will be automatically disabled if they cannot be used. All are either RSA or DSS cipher suites, meaning they need a certificate with an RSA or DSA key with its private key to be usable. If such a certificate with private key entry cannot be found in the keystore, it won't be usable by the KeyManager and SSLContext. Hence, they will be disabled when the handshake is actually attempted. This would typically result in an exception being thrown in the middle of the handshake on the server side ("javax.net.ssl.SSLHandshakeException: no cipher suites in common"), and the error messages you get on the client side via OpenSSL.