Uploading image to S3 and then resizing in node.js

Question

I am building a web-site where a user will upload an image and then the image has to ultimately be stored in Amazon S3. Now, given the possibility of uploading directly from browser, I considered the following options:

1. Resize to all three sizes in the browser and then upload to S3 directly - this works but the problem is that I am uploading multiple images, which I doubt is a good way to this.
2. Upload the original image to S3, resize in node.js - I was thinking of actually uploading only original image but then using node.js to resize.

What is the best way to do 2) so that I can minimize the footprint of the service I need to deploy.

Appreciate any pointers!

Cheers Vishal

Answer 1

i wouldn't do option 1 just because it's bad for user experience.

i suggest the following: only upload the original image to s3, then when a resized variant gets requested by a client, retrieve it from s3, resize, and deliver to the client through a CDN.

this beats resizing them all at once because:

• maybe you want to change the sizes later on. resizing on request lets you change sizes at will. otherwise, you would have to go through every image and redo all the resizes.
• the user doesn't have to wait until all the images are resized. instead, the user only waits for the image to be uploaded to s3 from the node.js server, which is fast assuming you're on aws.
• resizing all at once creates a memory spike. if you're on a low memory platform like heroku, you can hit the 512mb memory limit pretty quickly and your app will swap and become ridiculously slow. better to spread out the resizing operations.
• you only store the originals on S3. the resized versions are just derivatives - no need to store them.

i've written a library https://github.com/mgmtio/simgr to help me do this in my app.

Answer 2

G'day Vishal,

I imagine your largest images are going to be much larger than your smaller sizes, which are probably thumbnails or whatever. Because image file size more-or-less scales as the area of the image, this effect will be even more pronounced.

So any service you create which pulls the largest image back out of S3 to rescale it is likely to use more S3 bandwidth than the version which just uploads all three from the client! So it might be more sensible to just do option 1.

EDIT: If you really want to avoid uploading multiple files, I still wouldn't use the upload-directly-to-S3 option, since a) you'll have to detect new files on S3 and b) the total amount of data getting shipped around will still be greater than if you accept the image into your node app, resize it there and then save the various sizes to S3.

This question discusses image manipulation in Node.

-----Nick

Answer 3

i suggest using an Image CDN to Speed Up Image Delivery. Check here

Amazon S3 is Slow and Images are Heavy

Amazon S3 is known to be slow when serving web content directly. One reason why it is slow is that a bucket is only located in one geographical location. The location is selected when you create the bucket. For example, if your bucket is created on Amazon servers in California, but your users are in India, then images will still be served from California. This geographical distance causes slow image loading on your website.

Further, it is not uncommon to see very heavy images on S3, with large dimensions and high byte size. One can only speculate on the reasons, but it is probably related to the publication workflow and the convenience of S3 as a storage space.

using image delivery like ImageEngine while keeping S3’s workflow and convenience level.

Trust me when I say its very simple to implement... very very simple I repeat.

Once you do the nessary settings after creating account all you need is to Reference images using the ImageEngine domain

http://wq77sh2y.cdn.imgeng.in/path/image.jpg

OR

Reference images using the prefixing feature

http://wq77sh2y.cdn.imgeng.in/http://example.com/path/image.jpeg