12

We've recently moved to a new webhost that offers limited storage space so we're trying to move all of our user uploads (Avatars, Files, etc.) uploaded via the Paperclip gem to Amazon S3. I have several hundred files all corresponding to different models that I'm now trying to migrate en masse.

I found this document that introduces a nifty paperclip rake task:

rake paperclip:refresh:missing_styles

This command does some of the work for me, however, I noticed it's only setting up the files structure without sending any data - in addition it's not setting up any thumbnails defined using the :styles hash in the has_attached_file call. I.e., I have the following paperclip setup on one of my models:

class User < ActiveRecord::Base
  has_attached_file :avatar,
    :styles => {
      :thumb => "100x100#",
      :small  => "150x150>",
      :medium => "200x200" }
end

Here's some sample output after running the command:

$ rake paperclip:refresh:missing_styles
Regenerating User -> avatar -> [thumb, :small, :medium]
Regenerating Mercury::Image -> image -> [:medium, :thumb]
Regenerating Profile -> image -> [:home_feature, :large, :medium, :thumb]
Regenerating Page -> preview -> [:portfolio]
Regenerating Category -> default_image -> [:home_feature, :large, :medium, :thumb]

Navigating to my S3 Bucket I can see all the directories are correctly setup and for each attachment, but only for original image files and they're all 0 bytes. Am I misunderstanding the usage of this command? I couldn't find any other tool for uploading entire directories of files in bulk to S3, if there's a safe tool out there that already covers this without requiring payment then I'm open ears. I've tried building a ruby script to plug into their SDK and upload these files manually, but their Ruby documentation isn't great.

Noz
  • 6,216
  • 3
  • 47
  • 82
  • Does the Paperclip/S3 integration work for you at all? i.e. if you upload a brand new image on your app, does it end up on S3? The answer to that question is necessary to give any reasonable advice! – LAW Jan 03 '14 at 20:39
  • @LAW Beautifully. The only problem is trying to get already existing files there. – Noz Jan 03 '14 at 20:47

1 Answers1

22

Looks like I needed to go beyond ruby on this one, s3cmd seemed to be the most appropriate tool for this sort of job. In my case, the sync command did the trick:

s3cmd sync my-app/public/system/ s3://mybucket 
Noz
  • 6,216
  • 3
  • 47
  • 82
  • Brilliant. This has solved the whole asset sync nonsense for me. It's a shame I need half a dozen separate tools in order to make everything work (I'm SURE the asset precompile should do the work). – CJBrew Jan 05 '14 at 19:46