Set Expires header for an existing S3 object using AWS Java SDK

Question

I'm updating existing objects in an Amazon S3 bucket to set some metadata. I'd like to set the HTTP Expires header for each object to better handle HTTP/1.0 clients.

We're using the AWS Java SDK, which allows for metadata changes to an object without re-uploading the object content. We do this using CopyObjectRequest to copy an object to itself. The ObjectMetadata class allows us to set the Cache-Control, Content-Type and several other headers. But not the Expires header.

I know that S3 stores and serves the Expires header for objects PUT using the REST API. Is there a way to do this from the Java SDK?

Updated to indicate that we are using CopyObjectRequest

score 6 · Accepted Answer · answered Mar 21 '12 at 15:21

6

To change the metadata of an existing Amazon S3 object, you need to copy the object to itself and provide the desired new metadata on the fly, see copyObject():

By default, all object metadata for the source object are copied to the new destination object, unless new object metadata in the specified CopyObjectRequest is provided.

This can be achieved like so approximately (fragment from the top of my head, so beware):

AmazonS3 s3 = new AmazonS3Client();

String bucketName = "bucketName ";
String key = "key.txt";
ObjectMetadata newObjectMetadata = new ObjectMetadata();
// ... whatever you desire, e.g.:
newObjectMetadata.setHeader("Expires", "Thu, 21 Mar 2042 08:16:32 GMT");

CopyObjectRequest copyObjectRequest = new CopyObjectRequest()
                 .WithSourceBucketName(bucketName)
                 .WithSourceKey(key)
                 .WithDestinationBucket(bucketName)
                 .WithDestinationKey(key)
                 .withNewObjectMetadata(newObjectMetadata);

s3.copyObject(copyObjectRequest);

Please be aware of the following easy to miss, but important copyObject() constraint:

The Amazon S3 Acccess Control List (ACL) is not copied to the new object. The new object will have the default Amazon S3 ACL, CannedAccessControlList.Private, unless one is explicitly provided in the specified CopyObjectRequest.

This is not accounted for in my code fragment yet!

Good luck!

answered Mar 21 '12 at 15:21

Steffen Opel

63,899
11
192
211

I should have explained better that we are using `CopyObjectRequest` to do this now. I've updated the question to reflect this. – bkirkbri Mar 21 '12 at 16:14
The `ObjectMetadata.setHeader` is marked as internal use only. Have you used this successfully? Our code is only going to be run once, so we won't need to worry about it later if Amazon makes changes. But YMMV. – bkirkbri Mar 21 '12 at 16:15
Hmmh, I wasn't aware of that restriction indeed, but recall having used `Expires:` at some point a while back; I might be wrong though, insofar I often use other SDKs for interacting with S3 (e.g. C#/Python, which do definitely support this) and could have mixed that up - the code itself doesn't differ from the other `setXYZHeader()` methods currently (see [ObjectMetadata.java](https://github.com/amazonwebservices/aws-sdk-for-java/blob/master/src/main/java/com/amazonaws/services/s3/model/ObjectMetadata.java#L141)), so the restriction would be based on a non visible side effect, if any. – Steffen Opel Mar 21 '12 at 17:08
It's probably used by the other header methods internally and internal only because only certain headers will work. They should add a `setExpires` method on `ObjectMetadata` as it's the only header that doesn't have it's own method. – bkirkbri Mar 21 '12 at 17:52
@SteffenOpel What is the "right" date for Expires value? 2042 or 2014 or 2024? What is depends from this date? Is this a reason to set date to 2042 (or any far date)? – Dima Deplov Sep 05 '13 at 14:11
@flinth - there's no 'right' date (I just made that up back then), it entirely depends on your use case; you might want to read Yahoo's classic [Best Practices for Speeding Up Your Web Site](http://developer.yahoo.com/performance/rules.html) though, which explains one specific use case for implementing a _"Never expire" policy by setting far future Expires header_ for static components, see section _Add an Expires or a Cache-Control Header_ on the linked page for details - please also check [Controlling Freshness with the Expires HTTP Header](http://www.mnot.net/cache_docs/#EXPIRES) in case. – Steffen Opel Sep 05 '13 at 18:09
@SteffenOpel thanks for your reply, I will check this sources, right now I asked a question about cache-control and expires, maybe this links helps me, thank you. May be you'll find interesting my question. http://stackoverflow.com/questions/18642446/amazon-s3-cache-control-and-expiry-date-difference-and-setting-trough-rest-api – Dima Deplov Sep 05 '13 at 18:56

score 1 · Answer 2 · answered Mar 21 '12 at 09:13

1

We were looking for a similar solution and eventually settled for max-age cache-control directive. And we eventually realized that hte cache-control overrides the Expires even if expires is more restrictive. And anyways cache-control met our requirement as well.

answered Mar 21 '12 at 09:13

Anoop Halgeri

662
3
8
17

1

Thanks. You are absolutely correct `Cache-Control` supersedes the `Expires` header. Still, we'd like to include it for HTTP/1.0 clients that do not respect `Cache-Control`. – bkirkbri Mar 21 '12 at 16:03

Set Expires header for an existing S3 object using AWS Java SDK

2 Answers2

Linked