Does Amazon S3 support HTTP request with basic authentication

Question

I would like to set up an Amazon S3 account, create a bucket, upload some data, and that this data will be available using HTTP GET with basic authentication.

I know that there are several ways to get the S3 data authenticated (query string and such), but I would like to be able to provide a simple username/password scheme for authentication.

Is this possible?

Answer 1

This is now possible using CloudFront and Lambda@Edge (generally available since July 2017 in the us-east-1 region).

1. Create a S3 bucket
2. Setup a CloudFront distribution in-front of the bucket, restricting access to the bucket so that only CloudFront can access it directly
3. Create a Lambda function, which will mimic Basic HTTP Auth handshake with the browser. Assign it to the CloudFront Viewer Request behavior.

Here's the Lambda function: https://gist.github.com/lmakarov/e5984ec16a76548ff2b278c06027f1a4

Here's an article with more details: https://medium.com/@lmakarov/serverless-password-protecting-a-static-website-in-an-aws-s3-bucket-bfaaa01b8666

Answer 2

The short answer is no, not using basic auth. But here is a way that is effectively the same as basic auth, and that is easily than other solutions listed. I believe it is secure, but I don't know for sure.

You can set conditions on s3 buckets that match the headers on the request. As an example you can use the useragent, and referer headers as something equivalent to username and password in basic auth. Normally the useragent is the browser, and OS (like Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0), and the referer is the previous webpage.

Here is an example s3 bucket policy that allows putting objects, and getting objects by matching the useragent, and referer (note change: BUCKETNAME, USERNAME, PASSWORD, AWS_REGION, and FILENAME to your details):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "allow-username-and-password-access",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": [
                "s3:PutObject",
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::BUCKETNAME/*",
            "Condition": {
                "StringEquals": {
                    "aws:UserAgent": "USERNAME",
                    "aws:Referer": "PASSWORD"
                }
            }
        }
    ]
}

To put a resource in the bucket you can use a curl request like this (note change: BUCKETNAME, USERNAME, PASSWORD, AWS_REGION, and FILENAME):

curl --user-agent USERNAME --referer PASSWORD --upload-file "FILENAME" --request PUT "https://s3-AWS_REGION.amazonaws.com/BUCKETNAME/FILENAME"

To get use the resource you can use something like this:

curl --user-agent USERNAME --referer PASSWORD "https://s3-AWS_REGION.amazonaws.com/BUCKETNAME/FILENAME" > FILENAME

Once again, I believe this is secure, as the useragent, and referer should be encrypted if you are using https, but please tell me if it is not.

Answer 3

You can develop it yourself as a web app or a part of your existing application. It will consume HTTP requests, retrieve their URI component, convert it to S3 object name and use getObject() to get its content (using one of available S3 SDKs, for example AWS Java SDK).

Otherwise, you can try a hosted solution - s3auth.com (I'm a developer). It's an open source project, and you can see how this mechanism is implemented internally at one of its core classes. HTTP request is processed by the service and then re-translated to Amazon S3 internal authentication scheme:

This architecture diagram explains how the project is implemented. The PNG picture is loaded from Amazon S3 bucket maven.s3auth.com, which is not readable anonymously. Full URL of this image is

http://s3auth:s3auth@maven.s3auth.com/texry/packages.png

Check also this article: Basic HTTP Auth for S3 Buckets

Answer 4

By layering multiple AWS services you can achieve something close to Basic HTTP Authorisation.

1. Create a s3 static site.
2. Create a CloudFront Distribution to serve the s3 static site (use the static site url and not the bucket name)
3. Use AWS WAF to create a rule which only allows requests with the correct http Authorization header. This will be a sting match rule on the contents of the Authorization header.
4. Use Route53 to route a custom domain to the CloudFront Distribution

You should now have a static site which can only be accessed with the correct username and password.

NOTE: using this setup you will not be prompted for you credentials as the request is blocked with a 403 Forbidden instead of 401 Unauthorized.

NOTE: You can create a CloudFront distribution in front of a s3 bucket directly, but you will not be able to default to a root index file in sub-folders.

Answer 5

No this is not possible. You have to conform to Amazons Authentication API

Check out some of the of wrappers listed here.

Answer 6

By now, this has become relatively straightforward on AWS. It's basically 3 things that need to be done:

1. Create a CloudFront function to add Basic Auth into the request.
2. Configure the Origin of the CloudFront distribution correctly in a few places.
3. Activate the CloudFront function.

That's it, no particular bells & whistles otherwise. Here's what I've done:

First, go to CloudFront, then click on Functions on the left, create a new function with a name of your choice (no region etc. necessary) and then add the following as the code of the function:

function handler(event) {
  var user = "myuser";
  var pass = "mypassword";
  var requiredAuth = "Basic " + `${user}:${pass}`.toString('base64');

  var authHeader = event.request.headers.authorization;

  if (authHeader && authHeader.value === requiredAuth) {
    return event.request;
  } else {
    return {
      statusCode: 401,
      statusDescription: "Unauthorized",
      headers: {
        "www-authenticate": { value: 'Basic realm="Application"' },
      },
    };
  }
}

Then you can test with directly on the UI and assuming it works and assuming you have customized username and password, publish the function.

Next, open your CloudFront distribution and do the following:

1. Make sure your S3 bucket in the origin is configured as a REST endpoint and not a website endpoint, i.e. it must end on .s3.amazonaws.com and not have the word website in the hostname.
2. Also in the Origin settings, under "S3 bucket access", select "Yes use OAI (bucket can restrict access to only CloudFront)". In the setting below click on "Create OAI" to create a new OAI (unless you have an existing one and know what you're doing). And select "Yes, update the bucket policy" to allow AWS to add the necessary permissions to your OAI.
3. Finally, open your Behavior of the CloudFront distribution and scroll to the bottom. Under "Function associations", for "Viewer request" select "CloudFront Function" and select your newly created CloudFront function. Save your changes.

And that should be it. With a bit of luck a matter of a couple of minutes (realistically more, I know) and especially not additional complexity once this is all set up.

Answer 7

I myself was trying to find solution to this problem. This post here has listed them all. Quoting the lines:

I’ve been looking for months for a solution to add Basic HTTP Authentication to S3 buckets on Amazon. There are options involving pre-signed URLs (single object only), using a 3rd-party free or commercial service (privacy concerns), spinning up an EC2/Heroku/etc. with middleware to proxy requests (complicated and not serverless), using page redirects and bucket policies (not secure).

Bucket policies solution: I have personally tried this and it seems perfectly secure to me (unless you have a way to bypass aws bucket policies). It just requires s3 bucket to operate. Simple to implement. Basic idea:

1. Restrict access to the entire site, except allow public access to Entry File and Secret File.
2. Entry File secure.html that accepts an user input for the password and redirects to Secret File
3. Secret File thisisasecret that redirects to Main File (index.html) that hosts the real content of the site
4. Main File main.html that only allows access to requests originated from the same site.
5. All the other content like css, js files would be restricted by a bucket policy which will allow them served iff the request originated from your bucket URL.

Using aws Lambda@Edge: This solution require s3, aws lambda and aws cloudfront to operate. Basic idea:

1. Create a secure.html. Create text boxes to enter basic credentials from user here. This file should be publicly accessible and should call a lambda function.
2. While configuring cloudfront create a behaviour which says 'if you want to reach index.html, you need to do so via Signed URL'.
3. Just like above create a bucket policy to allow access to js, css, etc. files only when origin is your bucket URL.

Answer 8

Have a look at my answer here in this somewhat related question.

The question there was to get a bucket-object LIST without heavy S3 login authentication needed. So from your question, my answer does not really explain your inquiry for Http-request with basic username/password authentication - but rather gives a possibility to make the bucket private and still retrieve its data knowing an identity-pool (ID) and a Amazon Resource Name (ARN) (which you can consider similar to username and password). Of course, to get the (ID) and (ARN) you have to do some settings-work in the AWS main pages as described in my 17-step answer here... - and also, it is not part of the GET-request but rather given within the AWSS3 framework that Amazon provides. I hope, it still gives you more options to your question ;)

Answer 9

I'm from AdroitLogic. About the linked article, it shows how the UltraESB could be placed between your client and Amazon S3 to authenticate your requests. If required, it could create a "proxy" service which will accept basic authentication from your client, and send the credentials the way Amazon S3 expects. This could be done in a trivial manner, and will hide any complexity for your client.