96

I am new to writing Rails and APIs. I need some help with S3 storage solution. Here's my problem.

I am writing an API for an iOS app where the users login with the Facebook API on iOS. The server validates the user against the token Facebook issues to the iOS user and issues a temporary Session token. From this point the user needs to download content that is stored in S3. This content only belongs to the user and a subset of his friends. This user can add more content to S3 which can be accessed by the same bunch of people. I guess it is similar to attaching a file to a Facebook group...

There are 2 ways a user can interact with S3... leave it to the server or get the server to issue a temporary S3 token (not sure of the possibilities here) and the user can hit up on the content URLs directly to S3. I found this question talking about the approaches, however, it is really dated (2 yrs ago): Architectural and design question about uploading photos from iPhone app and S3

So the questions:

  • Is there a way to limit a user to access only some content on S3 when a temporary token is issued? How can I do this? Assume there's... say 100,000 or more users.
  • Is it a good idea to let the iOS device pull this content out directly?
  • Or should let the server control all content passing (this solves security of course)? Does this mean I have to download all content to server before handing it down to the connected users?
  • If you know rails... can I use paperclip and aws-sdk gems to achieve this kinda setup?

Apologies for multiple questions and I appreciate any insight into the problem. Thanks :)

Community
  • 1
  • 1
dineth
  • 9,822
  • 6
  • 32
  • 39
  • 1
    found this and thought i would comment for others looking http://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html – dibble Dec 11 '15 at 15:17

2 Answers2

114

Using the aws-sdk gem, you can get a temporary signed url for any S3 object by calling url_for:

s3 = AWS::S3.new(
  :access_key_id => 1234,
  :secret_access_key => abcd
)
object = s3.buckets['bucket'].objects['path/to/object']
object.url_for(:get, { :expires => 20.minutes.from_now, :secure => true }).to_s

This will give you a signed, temporary use URL for only that object in S3. It expires after 20 minutes (in this example), and it's only good for that one object.

If you have lots of objects the client needs, you'll need to issue lots of signed URLs.

Or should let the server control all content passing (this solves security of course)? Does this mean I have to download all content to server before handing it down to the connected users?

Note that this doesn't mean the server needs to download each object, it only needs to authenticate and authorize specific clients to access specific objects in S3.

API docs from Amazon: https://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html#RESTAuthenticationQueryStringAuth

ejdyksen
  • 1,489
  • 1
  • 12
  • 15
  • 3
    Thanks for that @ejdyksen. The solution I've devised used exactly that (I haven't updated the question with my answer)! So my solution was to do authenticated URLs for GET requests. However, when a user contributes content, he creates resources to a specific /bucket/user/objectname location using federated IAM token (temporary credentials that expire) with the policy attached to allow for /bucket/user/* write access. so no user in the system can do harm to other users' content. Seems to work okay. Appreciate your answer. – dineth Jun 06 '12 at 00:06
  • 1
    It's just a temporary url, not one time url. – petertc Jul 20 '14 at 12:26
  • 1
    @okwap you're right. "One time" is not very accurate, as it can be used an unlimited number of times before the expiration. I updated the answer to reflect that. – ejdyksen Jul 21 '14 at 14:54
  • 5
    If you're using v2 of the aws-sdk-ruby, note that the methods are somewhat different : http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Object.html#presigned_url-instance_method – vijucat May 04 '15 at 10:08
  • 2
    Isn't it a risk that the user can see my access key? There is also the secret key (converted) – user2503775 Sep 01 '15 at 07:58
  • Users will be able to see your AWS credentials by inspecting the link. Even if they are not technical, they can easily the authenticated URL on an S3 error page when something goes wrong. You can get around this by streaming/downloading the file to your server and then serving it to the client. – Dennis Jun 30 '16 at 10:08
  • 3
    @Dennis the whole point is that the file doesn't have to hit your server. The link does not contain your AWS credentials. It might contain the `ACCESS_KEY_ID` (I don't remember off the top of my head), but that is not intended to be a secret. – ejdyksen Jul 09 '16 at 03:02
  • 2
    @ejdyksen you're right, I just verified that the URL only contains the `AWS_ACCESS_KEY_ID`. I originally thought that `AWS_SECRET_ACCESS_KEY` was also displayed but it isn't. – Dennis Jul 13 '16 at 18:02
  • thanks @ejdyksen for this solution. I am working on a rails application and working on integrating s3 with rails application I have multiple module like images, profile_pics etc. all these object dont need to be protected but I have upload folder which container the files which I like to protect do you have idean how I can do this via s3? – Hitesh Ranaut Jul 18 '19 at 14:13
47

The above answers use the old aws-sdk-v1 gem rather than the new aws-sdk-resources version 2.

The new way is:

aws_resource = Aws::S3::Resource::new
aws_resource.bucket('your_bucket').object('your_object_key').presigned_url(:get, expires_in: 1*20.minutes)

where your_object_key is the path to your file. If you need to look that up, you would use something like:

s3 = Aws::S3::Client::new
keys = []
s3.list_objects(bucket: 'your_bucket', prefix: 'your_path').contents.each { |e| 
  keys << e.key
}

That information was startlingly difficult to dig up, and I almost just gave up and used the older gem.

Reference

http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Object.html#presigned_url-instance_method

Chris
  • 2,537
  • 1
  • 25
  • 22
chaqke
  • 1,497
  • 17
  • 23