2

I'm trying to save the images from a third-party API to my own S3 bucket using Node.js and MongoDB. The API provides a URL to the image on the third-party servers. I've never done this before but I'm assuming I have to download the image to my own server and then upload it to S3?

Should I save the image to mongodb with GridFS and then delete it once it is on S3? If so, what's the best way to do that?

I've read and re-read this previous question:

Problem with MongoDB GridFS Saving Files with Node.JS

But I can't find good documentation on how I should determine buffer/chunk size and other attributes for a JPEG image file.

Once I've saved the file on my server/database, then it seems like I should use:

https://github.com/appsattic/node-awssum

To upload it to S3. Is that a good idea?

I apologize if this is an obvious answer, I'm pretty green when it comes to databases and server scripting.

Community
  • 1
  • 1
kaxline
  • 63
  • 6
  • go with the default chunk size, unless the images are relatively small, in which case you can store them in a regular document as binary type. – Barrie May 23 '12 at 18:20

1 Answers1

-1

The easiest thing to do would be to save the image onto disk and then stream the upload from there using AwsSum's S3 PutObject operation. Alternatively, if you have the file contents in a Buffer you can just use that.

Have a look at the following two examples and they should help you figure out what to do:

Let me know if you need any more help. :)

Cheers, Andy

Disclaimer: I'm the author of AwsSum.

chilts
  • 451
  • 3
  • 12
  • hi andy, sorry for the giant delay in accepting this answer, i was not alerted that you had responded. it's actually sad, but i haven't worked on this issue since you responded so this is still relevant for me. thank you! – kaxline Feb 23 '13 at 18:42
  • 1
    The links are dead :( – Suraj Pathak Feb 09 '15 at 07:41
  • That's a huge bottleneck, you can just stream the request from the image location to s3. – Ian Elvister Apr 20 '21 at 20:52