4

I have an AWS Simple Storage Service (S3) bucket set as the origin for an AWS CloudFront CDN distribution.

When brand new images that never existed in the S3 bucket or CloudFront distribution (small, <50k) are uploaded to the S3 bucket, they are available almost instantly when requested from the S3 resource, but take over 10 minutes to become available from the CloudFront resource.

Once the images are available in CloudFront, they load quickly. The problem is that the images seem to take a long time to become available in CloudFront.

I have reduced TTL to 0 for Min, Max, and Default with no noticeable change.

  • Is this expected?

  • Is there anything I can do to speed this up?

Thank You

LWSChad
  • 331
  • 3
  • 14
  • 1
    Invalidating Objects - http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html ? –  Jan 06 '17 at 19:00
  • @Igor How is this applicable? The "new S3 content" I'm referring to is not "updated" images, but brand new images that never existed in the bucket or distribution. My previous understanding, and the understanding gleaned from the article, define invalidation as removing a resource from CloudFront so that it may be pulled from the origin again. – LWSChad Jan 06 '17 at 19:24
  • 1
    I am inclined to believe that this is a duplicate of [AWS CloudFront and S3: How to make new S3 content immediately available in CloudFront](http://stackoverflow.com/a/35541525/1695906). If you don't have your Error Cache TTL set to 0, that is going to be your problem. – Michael - sqlbot Jan 06 '17 at 22:14
  • BOOM! That does it. Sorry for the dupe, I followed every link Stack suggested, and that was not one of them. – LWSChad Jan 07 '17 at 00:53

1 Answers1

0

Yes, this is expected - no there's not much you can do about it unfortunately.

As @igor said, you can do an invalidation of the object. I'm not sure if this will actually speed it up, but you can try. You can do this by setting up events from S3 to Lambda and calling CloudFront to invalidate that object, or you can call it side by side with the upload call you're making - decoupling with Lambda is better but whatever.

All files that go to S3 would have to be available through S3 obviously, don't restrict bucket access when you put the CloudFront layer on. You can make the S3 GET request a bit faster by enable Accelerated Transfer, too (if you want a faster 'direct to S3' request, has no impact if you use CloudFront)

I would recommend doing a fail-over call that says:

function getFromS3()
    serveFileToUser()

function getFromCloudFrontEndpoint()
    if !exists
        getFromS3()
    else
        serveFileToUser()
Community
  • 1
  • 1
iSkore
  • 7,394
  • 3
  • 34
  • 59
  • Thanks! I'll experiment with something like that. – LWSChad Jan 06 '17 at 19:58
  • 1
    This is not correct. The comments indicate that the question is about **new** objects. There is no reason to expect any delay with correct configuration. *"Always treat CloudFront as an eventual consistency environment. It has to run those files all over the world."* No. CloudFront does not push content anywhere. CloudFront is a pull-through cache. When you request an object through an edge, the edge pulls the content from the origin -- in real time -- unless it's already cached at that edge because of a prior request (and not expired) -- and returns it. – Michael - sqlbot Jan 06 '17 at 22:22
  • 1
    Also, S3 Transfer Acceleration has no impact here. It's not obvious from the docs, but the S3 Transfer Acceleration feature is actually accomplished by S3, invisibly to the user, using geo-dns to route requests between browser and bucket over *CloudFront's* infrastructure (but without caching). CloudFront wouldn't benefit by running the requests through its infrastructure twice. – Michael - sqlbot Jan 06 '17 at 22:26