Can't access Buckets on S3 using AWS S3 Java API using HTTPS

Question

Trying to access our local S3 storage I first ran into a problem with a missing certificate:

Exception in thread "main" com.amazonaws.SdkClientException: Unable to execute HTTP request: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Switching the cert checking off:

System.setProperty(SDKGlobalConfiguration.DISABLE_CERT_CHECKING_SYSTEM_PROPERTY, "true");

I managed 'connecting' to the server. But this doesn't yield any results (aka an empty list), using the code below. Pretty simple so far.

Both the endpoint, as well as the bucket name should be correct, as I cut'n'pastedd them out of my (working) S3 browser.

Credentials seem to be correct, too, as I'm running in a 404 when I mess those up. Again - copied those from y S3 broswer.

Using the S3 browser, I can access the bucket, add files an so on.

When running List<Bucket> buckets = s3Client.listBuckets(); it simply returns an empty list - without any exception.

Console output is:

Juli 03, 2019 8:48:07 NACHM. com.amazonaws.http.AmazonHttpClient createSocketFactoryRegistry WARNUNG: SSL Certificate checking for endpoints has been explicitly disabled. Juli 03, 2019 8:48:08 NACHM. com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles WARNUNG: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code. Juli 03, 2019 8:48:08 NACHM. com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles WARNUNG: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code. Juli 03, 2019 8:48:23 NACHM. com.amazonaws.http.AmazonHttpClient createSocketFactoryRegistry WARNUNG: SSL Certificate checking for endpoints has been explicitly disabled.

Any advice?

System.setProperty(SDKGlobalConfiguration.DISABLE_CERT_CHECKING_SYSTEM_PROPERTY, "true");
        String accessKey = "myaccess";
        String secretKey = "mysecret";

        AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);
        ClientConfiguration clientConfig = new ClientConfiguration();
        clientConfig.setProtocol(Protocol.HTTPS);

        EndpointConfiguration endpointConfig = new EndpointConfiguration("endpoint.com", Regions.EU_CENTRAL_1.getName());
        AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
                .withCredentials(new AWSStaticCredentialsProvider(credentials))
                //.withRegion(defaultRegion)
                .withEndpointConfiguration(endpointConfig)
                .build();

        List<Bucket> buckets = s3Client.listBuckets();
        for (Bucket bucket : buckets) { 
                System.out.println(
                        bucket.getName() + 
                        "\t" +
                        StringUtils.fromDate(bucket.getCreationDate())
                );
        }

When running List<Bucket> buckets = s3Client.listBuckets(); it simply returns an empty list - without any exception.

There are (according to the S3 browser) 2 files in this bucket's root.

Console output is:

Juli 03, 2019 8:48:07 NACHM. com.amazonaws.http.AmazonHttpClient createSocketFactoryRegistry WARNUNG: SSL Certificate checking for endpoints has been explicitly disabled. Juli 03, 2019 8:48:08 NACHM. com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles WARNUNG: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code. Juli 03, 2019 8:48:08 NACHM. com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles WARNUNG: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code. Juli 03, 2019 8:48:23 NACHM. com.amazonaws.http.AmazonHttpClient createSocketFactoryRegistry WARNUNG: SSL Certificate checking for endpoints has been explicitly disabled.

Any advice?

Why are you setting an `EndpointConfiguration`? Also, is this code running on an Amazon EC2 instance or your own computer? Please note that it is recommended never to put credentials in your code. Instead, store them in a credentials file. (Use the [AWS Command-Line Interface (CLI)](http://aws.amazon.com/cli/) `aws configure` command to create the credentials file.) — John Rotenstein, Jul 04 '19 at 00:21
The code is - just for test and learning puposes - running on my machine. That's the reason/Excuse why I'm using inline credentials. — derOlli, Jul 04 '19 at 06:24
Why are you setting an `EndpointConfiguration`? You might want to look at some code examples on [Creating, Listing, and Deleting Amazon S3 Buckets - AWS SDK for Java](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/examples-s3-buckets.html). — John Rotenstein, Jul 04 '19 at 06:39
@JohnRotenstein Sorry - hit too soon. Full comment: The code is - just for test and learning puposes - running on my machine. That's the reason/Excuse why I'm using inline credentials. I'm trying to access a radowsgw instance within our company network, so no aws - but the libraries - involved. I started with the snippets taken from here [link](http://docs.ceph.com/docs/mimic/radosgw/s3/java/) Where's the problem about abbout using `EndpointConfiguration`? I am aware of the aws documentation, but am unsure on the regeion (my S3 shows "default") and https connections. — derOlli, Jul 04 '19 at 07:08
I don't know anything about `radowsgw`, but my guess is that if you are supplying an Endpoint, you wouldn't use a Region (since in AWS, the region determines the endpoint). — John Rotenstein, Jul 04 '19 at 07:28
@JohnRotenstein That is definitly different in my case, as I'm connection to an internal machine (either using the IP or the machines name). I'll give the AWS examples another try - any suggestion how to add the server to my cacerts file? And how to I get the right file (as I'm not connecting to AWS)? — derOlli, Jul 04 '19 at 07:39
Sorry, but I have no idea to do non-AWS connections. Maybe somebody else will be able to assist. — John Rotenstein, Jul 04 '19 at 07:41

score 3 · Answer 1 · answered Jul 12 '19 at 12:49

so I'm finally answering myself.

Disabling the endpoint verification using

System.setProperty(SDKGlobalConfiguration.DISABLE_CERT_CHECKING_SYSTEM_PROPERTY, "true");

works like a charm. The problem had 2 reasons:

My account didn't have the permissions listing the buckets (no idea why, won't happen when accessing an AWS S3).
I didn't supply the bucket name correctly, as I had to add a leading "/" to the bucket name.

Point 1 could be solved accessing an object listing:

ObjectListing objects = conn.listObjects("/my_bucket_name");

Point 2 could obviously solved by adding a leading "/" to the bucket name.

Cheers,

Olli

Can't access Buckets on S3 using AWS S3 Java API using HTTPS

1 Answers1