0

I have a high load django project with a lot of images in it (in django.db.models.ImageField). I'm looking for a solution with the following criteria:

  1. Images are stored in Rackspace CloudFiles (using django-cumulus)
  2. Images' thumbnails are stored in cloud too
  3. Thumbnails are generated at the same moment as source image changes (I don't want thumbnails to be generated on template render as most of django's thumbnails libraries do)

I would be grateful for your help.

rdodev
  • 3,164
  • 3
  • 26
  • 34
Antonio
  • 822
  • 1
  • 10
  • 19
  • What libraries have you tried so far? – rdodev Jan 13 '14 at 15:11
  • you might want to check this Q/A: http://stackoverflow.com/questions/20912041/changing-django-storages-backend-from-from-s3-to-cloudfiles-and-dealing-with-old – rdodev Jan 13 '14 at 15:25

1 Answers1

2

For (1) you should be able to use django-cumulus, I'm not aware of another working package for Rackspace CloudFiles.

For (2) set the STATICFILES_STORAGE setting to point to the class which handles static files on the cloud. In my case I store my user-uploaded static files on Amazon S3, and I extend the S3BotoStorage class imported this way:

from storages.backends.s3boto import S3BotoStorage

from django-storages. Find what the equivalent class is for the django-cumulus module and use/extend accordingly; storage classes are in here. The key is to set STATICFILES_STORAGE to point to it.

For (3) use your thumbnail generation library to "fetch" the image at the various thumbnail sizes you need when the source image changes. This will create them immediately (if they do not exist yet). (This applies for the libraries I have used, which are sorl-thumbnail and easy_thumbnails.

Side note: "asynchronous generation" of thumbnails (so that the initial request does not have to wait for all thumbnails to be generated immediately) is a popular requirement, and it is fairly documented for the easy_thumbnails library here. Should be straightforward to set up if you have celery already enabled in your architecture.

Joseph Victor Zammit
  • 14,760
  • 10
  • 76
  • 102
  • the Rackspace module for `django-storages` is outdated/deprecated. We recommend using a different solution. If you **must** use `django-storages`, it's adviced you extend it using libcloud. – rdodev Jan 13 '14 at 15:24
  • @rdodev Thanks, since it works well for Amazon S3 I've assumed it works well for CloudFiles; didn't use it myself. What is the alternative? – Joseph Victor Zammit Jan 13 '14 at 15:26
  • Check the comment section of this Question: http://stackoverflow.com/questions/20912041/changing-django-storages-backend-from-from-s3-to-cloudfiles-and-dealing-with-old – rdodev Jan 13 '14 at 15:31
  • 1
    No prob. Good answer. – rdodev Jan 13 '14 at 15:34
  • Thanks for your answer. But for (3) standart thumnail generation on the fly will be still enabled. so when thumbnail is not completely generated and server gets several http hits on target page, can it cause easy_thumbnails to generate same thumbnail many times simultaneously? – Antonio Jan 14 '14 at 10:42
  • 1
    @Antonio Good question. In my experience I prevented that from happening by adding thumbnail generation to the "upload" process. I.e. I do not return a response to the user until the file has been uploaded and thumbnails generated. The front-end gives a nice "waiting" message, so that the site does not seem unresponsive while it is doing the work for a couple of seconds. – Joseph Victor Zammit Jan 14 '14 at 11:18
  • Sounds like a plan) I'll try this! – Antonio Jan 14 '14 at 18:13
  • `django-stdimage` should work with `s3boto` as well, if not let me know. – codingjoe Feb 04 '14 at 14:31