6

I have been building web downloaders over the years (e.g., using Apache HTTPClient and recently JBrowser [1]). These have worked OK till recently when some sites result in certification errors. I do not understand the details, and I cannot find a simple tutorial for people who know relatively little about certificates (e.g., what one looks like and how it obtained or created). This is a request for a default explanation of the simplest case and how to fix it. Typical error:

[2020-02-17T09:38:24.249][Instance 1][Port 57129] Warning: Single GUI Threadiong is enabled, FPS should be slower
[2020-02-17T09:38:29.737][Instance 1][Port 57129] Feb 17, 2020 9:38:29 AM com.sun.webkit.network.URLLoader doRun
[2020-02-17T09:38:29.737][Instance 1][Port 57129] WARNING: Unexpected error
[2020-02-17T09:38:29.737][Instance 1][Port 57129] java.io.IOException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target: https://osf.io/search/?q=coronavirus
[2020-02-17T09:38:29.737][Instance 1][Port 57129]   at com.machinepublishers.jbrowserdriver.StreamConnection.exec(StreamConnection.java:369)
[2020-02-17T09:38:29.737][Instance 1][Port 57129]   at com.machinepublishers.jbrowserdriver.StreamConnection.getResponseCode(StreamConnection.java:449)
[2020-02-17T09:38:29.737][Instance 1][Port 57129]   at com.sun.webkit.network.URLLoader.receiveResponse(URLLoader.java:414)
...

I can access the URL through browsers (Firefox, Chrome) and get HTML which represents what I want, but cannot access this programmatically.

I have read several accounts of how to fix this (e.g. [2]), but they generally refer to "your Keystore" or "trust manager" as if everyone knows what these are. I am concerned that if I don't know what I am doing, I could break security. I don't know how I add sites to these or whether I even should.

I am on MACOSX and appear to have a binary file

"/Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/JRE/lib/security/cacerts"

Some of the answers suggest I should have a file called "truststore.jks" but don't say where this should be or how it was created.

So I am asking for a simple explanation of the system components and the simplest way to fix it. In some cases, I can avoid it (e.g., by using curl from the command-line), so I don't know how much this is a Java (8) problem.

EDIT: Trust Store vs Key Store - creating with keytool seems to explain the difference between KeyStore and TrustStore, but I still don't have insight into what to do.

[1] http://machinepublishers.github.io/jBrowserDriver/com/machinepublishers/jbrowserdriver /JBrowserDriver.html [2] Using a custom truststore in java as well as the default one

greendino
  • 416
  • 3
  • 17
peter.murray.rust
  • 37,407
  • 44
  • 153
  • 217

1 Answers1

3

Sorry, I'm not sure how much detail you need. Hope it will get you started. I can expand on the steps as you need. Just let me know.

Keystore stores private key and public cert and truststore store trusted certificates for other services you want to trust.Java bundles the truststore cacerts and it contains default, well known trusted certificate authorities.

As part of the ssl handshake the service presents its public certificate from its keystore to the client and client on the other hand verifies the presented certificate against its truststore.

For any programmatic access I would create a separate truststore file and import the certificates for services you trust. So in your case you have to import the certificate that is presented by https://osf.io/search/?q=coronavirus.

Once you have created the truststore you can reference truststore using jvm args.

Steps

  1. Use KeyTool to create empty truststore
  2. Use InstallCert utility or from browser to import the cert into truststore
  3. Reference the newly created truststore in the jvm arg (javax.net.ssl.truststore)
s7vr
  • 73,656
  • 11
  • 106
  • 127
  • Thanks - this is a useful start. To review: I import OSF certificate into my cacert. Then when I ask OSF for a service it will ask for the certificate. What does this prove? That I have downloaded the certificate at some time? How does it help security?Does it prevent Man In the Middle? These all show the depth of my ignorance. – peter.murray.rust Feb 20 '20 at 20:19
  • Osf had its server certificate issued by a CA, their certificate is not present in your cacerts. If you trust the CA, then import their certificate. Then whenever osf changes certificates or servers you have a way to check, by the trusted third party. – bbaassssiiee Feb 20 '20 at 23:56
  • Service doesn’t ask for certificate in a one way ssl. Service presents its certificate for client to trust. It proves that some certificate authority has vouched for the service you are trying to access. More here https://en.m.wikipedia.org/wiki/Certificate_authority. As long as private keys are secure and not shared I don’t think man in the middle attack are possible. – s7vr Feb 21 '20 at 00:08