Using Cloudfront with Django S3Boto

Question

I have successfully set up my app to use S3 for storing all static and media files. However, I would like to upload to S3 (current operation), but serve from a cloudfront instance I have set up. I have tried adjusting settings to the cloudfront url but it does not work. How can I upload to S3 and serve from Cloudfront please?

Settings

AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME

DEFAULT_FILE_STORAGE = 'app.custom_storages.MediaStorage'
STATICFILES_STORAGE = 'app.custom_storages.StaticStorage'

STATICFILES_LOCATION = 'static'
MEDIAFILES_LOCATION = 'media'

STATIC_URL = "https://s3-eu-west-1.amazonaws.com/app/%s/" % (STATICFILES_LOCATION)
MEDIA_URL = "https://%s/%s/" % (AWS_S3_CUSTOM_DOMAIN, MEDIAFILES_LOCATION)

custom_storages.py

from django.conf import settings
from storages.backends.s3boto import S3BotoStorage

class StaticStorage(S3BotoStorage):
    location = settings.STATICFILES_LOCATION

class MediaStorage(S3BotoStorage):
    location = settings.MEDIAFILES_LOCATION

Do you have additional info why it does not work? Any error message? — Edwin Lunando, Jul 11 '15 at 14:24
This has already been answered here: http://stackoverflow.com/questions/8688815/django-compressor-how-to-write-to-s3-read-from-cloudfront/8888930#8888930 — Krzysztof Szularz, Jul 15 '15 at 13:53
In case it helps anyone, I also had to add the setting `AWS_DEFAULT_ACL = ''` for `collectstatic` to work and not return 403. — Danra, Aug 16 '15 at 12:15

Mark Galloway · Accepted Answer · 2015-07-16T03:35:23.110

36

Your code is almost complete except you are not adding your cloudfront domain to STATIC_URL/MEDIA_URL and your custom storages.

In detail, you must first install the dependencies

pip install django-storages-redux boto

Add the required settings to your django settings file

INSTALLED_APPS = (
    ...
    'storages',
    ...
)

AWS_STORAGE_BUCKET_NAME = 'mybucketname'
AWS_CLOUDFRONT_DOMAIN = 'xxxxxxxx.cloudfront.net'
AWS_ACCESS_KEY_ID = get_secret("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = get_secret("AWS_SECRET_ACCESS_KEY")

MEDIAFILES_LOCATION = 'media'
MEDIA_ROOT = '/%s/' % MEDIAFILES_LOCATION
MEDIA_URL = '//%s/%s/' % (AWS_CLOUDFRONT_DOMAIN, MEDIAFILES_LOCATION)
DEFAULT_FILE_STORAGE = 'app.custom_storages.MediaStorage'

STATICFILES_LOCATION = 'static'
STATIC_ROOT = '/%s/' % STATICFILES_LOCATION
STATIC_URL = '//%s/%s/' % (AWS_CLOUDFRONT_DOMAIN, STATICFILES_LOCATION)
STATICFILES_STORAGE = 'app.custom_storages.StaticStorage'

Your custom storages need some modification to present the cloudfront domain for the resources, instead of the S3 domain:

from django.conf import settings
from storages.backends.s3boto import S3BotoStorage

class StaticStorage(S3BotoStorage):
"""uploads to 'mybucket/static/', serves from 'cloudfront.net/static/'"""
    location = settings.STATICFILES_LOCATION

    def __init__(self, *args, **kwargs):
        kwargs['custom_domain'] = settings.AWS_CLOUDFRONT_DOMAIN
        super(StaticStorage, self).__init__(*args, **kwargs)

class MediaStorage(S3BotoStorage):
"""uploads to 'mybucket/media/', serves from 'cloudfront.net/media/'"""
    location = settings.MEDIAFILES_LOCATION

    def __init__(self, *args, **kwargs):
        kwargs['custom_domain'] = settings.AWS_CLOUDFRONT_DOMAIN
        super(MediaStorage, self).__init__(*args, **kwargs)

And that is all you need, assuming your bucket and cloudfront domain are correctly linked and the user's AWS_ACCESS_KEY has access permissions to your bucket. Additionally, based on your use case, you may wish to make your s3 bucket items read-only accessible by everyone.

edited Jul 16 '15 at 03:35

answered Jul 15 '15 at 20:24

Mark Galloway

4,050
19
26

Great, concise answer. – Danra Aug 12 '15 at 10:51
1

May I ask why `MEDIA_URL = '//%s/%s/` instead of `MEDIA_URL='http://%2/%s/` is preferred? – eugene Jan 15 '16 at 10:50
1

@eugene By omitting the protocol specifier (`//` rather than `http://` or `https://`) you allow the browser to automatically choose the proper protocol (http or https) according to how the containing document was loaded. So, you can serve both secure and non-secure pages and have the static links stay the same. See for example: http://stackoverflow.com/questions/4831741/can-i-change-all-my-http-links-to-just – Myk Willis Feb 04 '16 at 15:51
`STATIC_ROOT` and `MEDIA_ROOT` do not seem to be used with `django-storages`. – Myk Willis Feb 04 '16 at 16:07
@Mark Galloway Thanks a lot. – CrazyGeek Jan 09 '17 at 03:09
Does the Django Storages component handle CloudFront invalidation? – Alex Leith Mar 24 '17 at 04:31
@AlexLeith No. I haven't used Cloudfront in a while, but from what I recall invalidation is not really ideal and costs money. It's usually simpler to just point your clients to a new url... – Mark Galloway Mar 24 '17 at 17:21
Just a note to anyone on Python3. I ran "pip install django-storages-redux boto" while django-storages was installed, and I had issues with not being able to access storages in python and having to destroy my virtualenv and recreate it. A simple pip install -r requirements.txt did not work. – MagicLAMP Nov 17 '17 at 04:06
For those wondering why `django-storages-redux` vs `django-storages` you can find the answer [here](https://pypi.org/project/django-storages-redux/) under "Why fork?". On that note if you already have `django-storages` installed it should work. Also you might want to watch out that "read only" dows not block collectstatic. Lastly, the reason for the custom classes is to prevent users from overwriting existing static files, media file uploads should be placed in a different subfolder in the bucket. – Josh Jul 22 '20 at 13:28

score 3 · Answer 2 · answered Dec 15 '16 at 12:51

3

I had a similar issue and just setting AWS_S3_CUSTOM_DOMAIN to the Cloudfront url in Django's settings.py worked for me. You can check the code here.

answered Dec 15 '16 at 12:51

Rafay

6,108
11
51
71

would you need to enable PUT DELETE etc. options on the cloudfront side? Also, your link is dead. – David Schumann Mar 09 '20 at 17:24

Using Cloudfront with Django S3Boto

2 Answers2

Linked