Check if a "folder" exists in S3

Question

I'm having a bit of trouble with this code:

public boolean isExists(String key) {
    try {
        this.s3Client.getObjectMetadata(this.bucketName, key);
        return true;
    } catch (AmazonServiceException var3) {
        return false;
    }
}

It always returns false, even if the "folder" in S3 exists (either empty or non-empty), what could be wrong?

The s3Client in the code above is just a simple AmazonS3 client:

AmazonS3 s3client = AmazonS3ClientBuilder
    .standard()
    .withCredentials(new AWSStaticCredentialsProvider(credentials))
    .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(objectStoreEndpoint, objectStoreRegion))
    .build();

What's the proper way to check if folder exists in S3?

Does this answer your question? [How to check if bucket already exists in AWS S3](https://stackoverflow.com/questions/62940476/how-to-check-if-bucket-already-exists-in-aws-s3) — jjbskir, Jun 14 '23 at 16:21

score 7 · Accepted Answer · answered Jan 04 '21 at 23:11

7

The solution to this is to do:

public boolean isExists(String key) {
    ListObjectsV2Result result = this.s3Client.listObjectsV2(this.bucketName, key);
    return result.getKeyCount() > 0;
}

An empty "folder" (virtually speaking) would return key count of 1.

answered Jan 04 '21 at 23:11

quarks

33,478
73
290
513

1

Important note, ensure that `key` will end with `/`. If won't, then files with the same beginning will be taken into account as well. E.g. dir `abc`, file `abcd`. Once passing `abc` in key, `abcd` file will be taken into account. – kkochanski Oct 14 '21 at 09:26

score 2 · Answer 2 · answered Nov 16 '21 at 03:57

The method listObjectsV2(bucketName, key) in the example provided by quarks is deprecated, this is how it should be now:

  public boolean doesKeyExist(String bucket, String key) {
    ListObjectsV2Response listObjects =
        s3Client.listObjectsV2(ListObjectsV2Request.builder().bucket(bucket).prefix(key).build());
    return !listObjects.contents().isEmpty();
  }

score 0 · Answer 3 · answered Jan 04 '21 at 22:57

0

S3 doesn't have folders, it only has buckets and objects. Most developers use the / character to semantically organize object names, but S3 doesn't have a directory structure the way that filesystems do. In particular, there can never be such a thing as an "empty directory" on S3.

The closest you can do is to list based on a prefix.

answered Jan 04 '21 at 22:57

chrylis -cautiouslyoptimistic-

75,269
21
115
152

How do you list with prefix and check if there are items? – Fireburn Jan 04 '21 at 23:06

score 0 · Answer 4 · answered Aug 23 '23 at 17:19

/**
 * Check if it exists at least 1 key in the bucket beginning with the given prefix. Functionally, it checks if a S3 "folder" exists.
 * The given input (the prefix) S3 key must end with "/" to avoid accidentally handling "abcd" S3 object whereas we want to handle objects with "abc/" prefix (i.e. in the "abc" folder).
 *
 * @param bucket the S3 Bucket name
 * @param prefix the prefix (can be considered as the S3 "folder"). Must end with "/"
 * @return if the folder exists <=> if there is at least 1 S3 object with this prefix
 */
public boolean prefixOrFolderExists(String bucket, String prefix) {
    if (!prefix.endsWith("/")) {
        throw new IllegalArgumentException("We want to check if a S3 folder exists. The S3 prefix should end with '/'.");
    }
    // just retrieve 1 key maximum with '.maxKeys(1)'. It is enough to check if the folder exists and it will save some bytes in the network.  
    ListObjectsV2Response response = s3Client.listObjectsV2(b -> b.bucket(bucket).prefix(prefix).maxKeys(1));
    return response.keyCount() > 0;
}

Check if a "folder" exists in S3

4 Answers4