2

I'm trying to upload large files to Amazon S3 without using credentials. I'm creating a plugin for Octoprint with this, and I can't put any sort of credentials into the code due to it being public. Currently my code for uploads looks like this:

import boto3
from botocore import UNSIGNED
from botocore.client import Config

s3 = boto3.client('s3', config=Config(signature_version=UNSIGNED))

# Create an S3 client


filename = 'file.txt'
bucket_name = 'BUCKET_HERE'

s3.upload_file(filename, bucket_name, filename)

However, it gives me the following error:

S3UploadFailedError: Failed to upload largefiletest.mp4 to BUCKETNAMEHERE/largefiletest.mp4: An error occurred (AccessDenied) when calling the CreateMultipartUpload operation: Anonymous users cannot initiate multipart uploads.  Please authenticate.

Is there any way to work around this, or are there any suggestions for alternative libraries? Anything is appreciated.

Wolfgang Fahl
  • 15,016
  • 11
  • 93
  • 186
Jacob Paniagua
  • 21
  • 3
  • 2
    Store the credentials in a separate file, and modify the code to read the credentials from the file. Anyone using the plugin would have to create their own credentials file. – John Gordon Sep 02 '18 at 03:18

1 Answers1

1

Do you mean that the repository is public but the runtime environment is private? If so, the standard practice is to set environment variables like this:

# first pip install environ
import environ
SOME_KEY = env('SOME_KEY', default='')

This way, you can easily update your credentials without changing your code or compromising security.

Edit: Then on the machine this code will be run, you can set the environment variables as such:

Zilong Li
  • 889
  • 10
  • 23
  • It has to be a specific key for all users, though. A couple ideas I had would be to keep the keys encrypted, or somehow have the code retrieve a file with the keys off of a separate server for use in the code. I honestly don't care at all if someone gets a hold of those keys, they only have upload permission anyways. But it's against the AWS TOS to post the keys publically, so I have to find some sort of workaround. – Jacob Paniagua Sep 02 '18 at 14:59
  • Do users have touch the keys directly or they just need to upload things to S3? If it's the latter, then you can implement the uploading in the backend without exposing the keys. – Zilong Li Sep 03 '18 at 00:03
  • I'm not entirely sure how I would do that... To be clear, I don't have any actual servers of my own. Just the S3 subscription. Sorry if this is a stupid question, I'm an absolute newbie at this and all your help is appreciated. – Jacob Paniagua Sep 03 '18 at 01:13
  • In that case, you could probably bury the keys in the code somewhere and use an obscure variable name... ;) If the keys aren't that critical to security, I guess that'd be ok. – Zilong Li Sep 03 '18 at 02:33