5

I'm trying to achieve direct to Amazon S3 upload in Rails using jQuery-File-Upload and the aws-sdk gem, and following heroku's direct to S3 upload instructions. This is the upload form produced in the html:

<form id="pic-upload"
class="directUpload" 
data-form-data="{
"key":"uploads/59c99e44-6bf2-4937-9680-02c839244b33/${filename}",
"success_action_status":"201",
"acl":"public-read",
"policy":"eyJle...In1dfQ==",
"x-amz-credential":"AKIAJCOB5HQVW5IUPYGQ/20160101/us-east-1/s3/aws4_request",
"x-amz-algorithm":"AWS4-HMAC-SHA256",
"x-amz-date":"20160101T010335Z",
"x-amz-signature":"0f32ae...238e"}" 
data-url="https://websmash.s3.amazonaws.com" 
data-host="websmash.s3.amazonaws.com"
enctype="multipart/form-data"
action="/users/bazley/update_pictures" 
accept-charset="UTF-8" 
method="post">

This is the corresponding jQuery:

$(function() {
  $('.directUpload').find("input:file").each(function(i, elem) {
    var fileInput    = $(elem);
    var form         = $(fileInput.parents('form:first'));
    var submitButton = form.find('input[type="submit"]');
    var progressBar  = $("<div class='bar'></div>");
    var barContainer = $("<div class='progress'></div>").append(progressBar);
    fileInput.after(barContainer);
    fileInput.fileupload({
      fileInput:       fileInput,
      url:             form.data('url'),
      type:            'POST',
      autoUpload:       true,
      formData:         form.data('form-data'),
      paramName:        'file', // S3 does not like nested name fields i.e. name="user[avatar_url]"
      dataType:         'XML',  // S3 returns XML if success_action_status is set to 201
      replaceFileInput: false,
      progressall: function (e, data) {
        var progress = parseInt(data.loaded / data.total * 100, 10);
        progressBar.css('width', progress + '%')
      },
      start: function (e) {
        submitButton.prop('disabled', true);
        progressBar.
          css('background', 'green').
          css('display', 'block').
          css('width', '0%').
          text("Loading...");
      },
      done: function(e, data) {
        submitButton.prop('disabled', false);
        progressBar.text("Uploading done");
        // extract key and generate URL from response
        var key   = $(data.jqXHR.responseXML).find("Key").text();
        var url   = '//' + form.data('host') + '/' + key;
        // create hidden field
        var input = $("<input />", { type:'hidden', name: fileInput.attr('name'), value: url })
        form.append(input);
      },
      fail: function(e, data) {
        submitButton.prop('disabled', false);
        progressBar.
          css("background", "red").
          text("Failed");
      }
    });
  });
});

Trying to upload a file produces these logs:

Started POST "/users/bazley/update_pictures" for ::1 at 2016-01-01 21:26:59 +0000 Processing by CharactersController#update_pictures as HTML
Parameters: {
    "utf8"=>"✓", 
    "authenticity_token"=>"rvhu...fhdg==",
    "standardpicture"=>{
        "picture"=>#<ActionDispatch::Http::UploadedFile:0x0000010b32f530 
            @tempfile=#<Tempfile:/var/folders/19/_vdcl1r913g6fzvk1l56x4km0000gn/T/RackMultipart20160101-49946-7t94p.jpg>, 
            @original_filename="europe.jpg", 
            @content_type="image/jpeg", 
            @headers="Content-Disposition: form-data; name=\"standardpicture[picture]\"; filename=\"europe.jpg\"\r\nContent-Type: image/jpeg\r\n">
    }, 
    "commit"=>"Upload pictures", 
    "callsign"=>"bazley"
}

The form submits successfully, but it isn't working because Rails doesn't save the correct location ("picture", a string) on S3; instead it thinks the location is

"picture"=>#<ActionDispatch::Http::UploadedFile:0x0000010b32f530

You can see this in the submitted parameters. It should be something like:

"picture"=>"//websmash.s3.amazonaws.com/uploads/220f5378-1e0f-4823-9527-3d1170089a49/europe.jpg"}, "commit"=>"Upload pictures"}

What I don't understand is why it's getting the parameters wrong when all the correct information seems to be present in the form. It clearly says

data-url="https://websmash.s3.amazonaws.com" 

in the form, and the jQuery includes

url:  form.data('url'),

so what's going wrong?

For completeness: in the controller:

before_action :set_s3_direct_post
.
.
def set_s3_direct_post
  @s3_direct_post = S3_BUCKET.presigned_post(key: "uploads/#{SecureRandom.uuid}/${filename}", success_action_status: '201', acl: 'public-read')
end

The form:

<%= form_for :standardpicture, url: update_pictures_user_path,
             html: {  id: "pic-upload", class: "directUpload",
                      data: { 'form-data' => (@s3_direct_post.fields),
                              'url' => @s3_direct_post.url,
                              'host' => URI.parse(@s3_direct_post.url).host } 
                   } do |f| %>
  <div class="field">
    <%= f.label :picture %>
    <%= f.file_field :picture, accept: 'image/jpeg,image/gif,image/png' %>
  </div>
  <%= f.submit "Upload pictures", class: "btn btn-primary" %>
<% end %>

aws.rb initializer:

Aws.config.update({
  region: 'us-east-1',
  credentials: Aws::Credentials.new(ENV['AWS_ACCESS_KEY_ID'], ENV['AWS_SECRET_ACCESS_KEY']),
})
S3_BUCKET = Aws::S3::Resource.new.bucket(ENV['S3_BUCKET'])

EDIT

The console shows this error:

Uncaught TypeError: Cannot read property 'innerHTML' of null

inside this file (tmpl.self-c210...9488.js?body=1):

(function ($) {
    "use strict";
    var tmpl = function (str, data) {
        var f = !/[^\w\-\.:]/.test(str) ? tmpl.cache[str] = tmpl.cache[str] ||
                tmpl(tmpl.load(str)) :
                    new Function(
                        tmpl.arg + ',tmpl',
                        "var _e=tmpl.encode" + tmpl.helper + ",_s='" +
                            str.replace(tmpl.regexp, tmpl.func) +
                            "';return _s;"
                    );
        return data ? f(data, tmpl) : function (data) {
            return f(data, tmpl);
        };
    };
    tmpl.cache = {};
    tmpl.load = function (id) {
        return document.getElementById(id).innerHTML;
    };
    tmpl.regexp = /([\s'\\])(?!(?:[^{]|\{(?!%))*%\})|(?:\{%(=|#)([\s\S]+?)%\})|(\{%)|(%\})/g;
    tmpl.func = function (s, p1, p2, p3, p4, p5) {
        if (p1) { // whitespace, quote and backspace in HTML context
            return {
                "\n": "\\n",
                "\r": "\\r",
                "\t": "\\t",
                " " : " "
            }[p1] || "\\" + p1;
        }
        if (p2) { // interpolation: {%=prop%}, or unescaped: {%#prop%}
            if (p2 === "=") {
                return "'+_e(" + p3 + ")+'";
            }
            return "'+(" + p3 + "==null?'':" + p3 + ")+'";
        }
        if (p4) { // evaluation start tag: {%
            return "';";
        }
        if (p5) { // evaluation end tag: %}
            return "_s+='";
        }
    };
    tmpl.encReg = /[<>&"'\x00]/g;
    tmpl.encMap = {
        "<"   : "&lt;",
        ">"   : "&gt;",
        "&"   : "&amp;",
        "\""  : "&quot;",
        "'"   : "&#39;"
    };
    tmpl.encode = function (s) {
        /*jshint eqnull:true */
        return (s == null ? "" : "" + s).replace(
            tmpl.encReg,
            function (c) {
                return tmpl.encMap[c] || "";
            }
        );
    };
    tmpl.arg = "o";
    tmpl.helper = ",print=function(s,e){_s+=e?(s==null?'':s):_e(s);}" +
        ",include=function(s,d){_s+=tmpl(s,d);}";
    if (typeof define === "function" && define.amd) {
        define(function () {
            return tmpl;
        });
    } else {
        $.tmpl = tmpl;
    }
}(this));
Bazley
  • 2,699
  • 5
  • 36
  • 61
  • have you make it working ? can I see a live demo please – medBouzid Jun 15 '16 at 13:47
  • I posted my full solution [here](http://stackoverflow.com/a/36237811/346613), I hope it's useful. – Bazley Jun 15 '16 at 21:13
  • Thank you it does! before following your solution I was able to get it working using the same tutorial you followed, there is one thing needed, do you know how to delete an uploaded file (using ajax request) ? – medBouzid Jun 16 '16 at 12:27
  • I've updated the answer and added post deletion, take a look. – Bazley Jun 21 '16 at 13:42

2 Answers2

5

Finally found the answer here. Simply had to go to application.js and change

//= require jquery-fileupload

to

//= require jquery-fileupload/basic

Christ on a tandem. Just pissed away 50 rep points on getting a whole 2 more views.

Community
  • 1
  • 1
Bazley
  • 2,699
  • 5
  • 36
  • 61
0

This code uses aws-sdk-v2 for ruby, it's been submitted to Heroku to update their documentation on s3 direct uploads (https://devcenter.heroku.com/articles/direct-to-s3-image-uploads-in-rails#pre-signed-post)

Some aspects of this are not necessary, i.e. storing your own photo object in the DB after uploading to s3. And the code is obviously not as good as it could be, this was just to get it working. I'm happy to hear any suggestions to improve it or know if something here isn't working for you.

The Sidekiq job has a method do_work which is a class I use defined in BaseJob to give me some additional metrics and info you can't get without SidekiqPro, so you would need to change this to "perform" if you're not doing anything special with the Sidekiq perform class.

#ROUTE CODE
post 'create_photo' => 'users#create_photo'

#CONTROLLER CODE

#intial view, might be update or another method in your case
def new
  @user = current_user.find params[:user_id]            

  @s3_direct_post = Aws::S3::Bucket.new(name: ENV['aws_bucket']).presigned_post(key: "musea_upload/harrisjb/${filename}", success_action_status: "201", acl: "public-read") 
end

#called from ajax method in code below
# I store a photograph object with an id and s3 url in my DB
# this code calls a sidekiq job in the user model to create 
# the photo object in the DB after the photo is uploaded directly to s3
# not essential, but helpful to reference uploaded photos

def create_photo                                                              
  @user = current_user.find params[:user_id]            
  options = {}                                                                
  options['user_id'] = params[:user_id]                                     
  options['key'] = params[:key]                                               
  options['url'] = params[:url]                                               
  @user.create_photograph_from_s3(options)                                   

  respond_to do |format|                                                      
    format.js { }                                                             
   end                                                                         
end  

#MODEL CODE
def create_photograph_from_s3(options)
  CreatePhotographJob.perform_async(options)
end

#JOB CODE
  def do_work(options)                                                          
    user_id = options['user_id']                                              
    user = User.find_by(id: user_id)                                                
    Rails.logger.info "CREATE PHOTOGRAPH JOB FOR USER #{user_id}"             
    url = options['url'].gsub("//h", "h")                                       
    user.create_photograph_from_file(url)                                
    Rails.logger.info "PHOTOGRAPH CREATED"                                      
  end 


#VIEW CODE
<%= form_for(:user_avatar, :remote => true, :url => p_user_avatar_upload_to_s3_path(user_id: @user.id), html: { id: "uploader", class: "uploader white-bg" }) do |f| %>                                                                                
  <input name="authenticity_token" type="hidden" value="<%= form_authenticity_token %>">
  <%= f.file_field :photo, multiple: true, style: 'margin-top: 20px; height: 5em; width: 100%; padding-top: 5%; padding-left: 20%;', class: 'form-group form-control light-gray-bg' %>                                       
<% end %>                         

# JS -- if you want this in your view use the content_for
# or you can store the functions in application.js or wherever you want

<% content_for :javascript do %>
  <script>  
      $(function() {
        $('.uploader').find("input:file").each(function(i, elem) {
          var fileInput    = $(elem);
          var form         = $(fileInput.parents('form:first'));
          var photos       = $('#photo-list');
          var photo_errors = $('#photo-list-errors');
          var photo_name   = 'photo_' + i;
          var submitButton = form.find('input[type="submit"]');

          //taken from heroku example, some bootstrap specific CSS here as well
          var progressBar  = $("<div class='progress-bar progress-bar-success progress-bar-striped active style='width: 0%;'></div>");
          var barContainer = $("<div class='progress col-md-12 col-sm-12 col-xs-12' style='margin-top: 20px; padding-left: 0px; padding-right: 0px;'></div>").append(progressBar);
          fileInput.after(barContainer);

          //jquery direct upload do its thing
          fileInput.fileupload({
            url:             '<%= @s3_direct_post.url %>',
            type:            'POST',
            autoUpload:       true,
            paramName:        'file', // S3 does not like nested name fields i.e. name="user[avatar_url]"
            dataType:         'XML',  // S3 returns XML if success_action_status is set to 201
            replaceFileInput: false,
            formData:         <%= @s3_direct_post.fields.to_json.html_safe %>,
            fileInput:       fileInput,
            progressall: function (e, data) {
              var progress = parseInt(data.loaded / data.total * 100, 10);
              progressBar
                .text(progress + '%')
                .css('role', 'progressbar')
                .css('width', progress + '%');

            },

            start: function (e) {
              $('.spinner').removeClass('hidden');
              $('.progress-bar').addClass('progress-bar-striped');
              submitButton.prop('disabled', true);
            },

            // Called when all files are uploaded
            stop: function (e) {
              $('.progress-bar').removeClass('progress-bar-striped');
              $('.progress-bar').html('Done!');
            },

            //
            done: function(e, data) {
              $('.spinner').addClass('hidden');
              submitButton.prop('disabled', false);

              // extract key and generate URL from response
              var key   = $(data.jqXHR.responseXML).find("Key").text();
              var url   = '//<%= @s3_direct_post.url %>/' + key;

              // when uploading entire folders dropped onto file_field
              // ignore .DS_Store (common with iPhone photos, etc)    
              if (key.indexOf(".DS_Store") >= 0 ) {
                console.log("nope. not doing it.");

              } else {

                // this is showing a preview photo on the left side of the screen
                // as photos are being uploaded

                  photos.prepend("<div class='photo col-md-2 col-sm-3 col-xs-3' style='margin-bottom: 10px; margin-top: 10px; background-image: url(<%= @s3_direct_post.url %>/" + key + ");  background-position: 50% 50%; background-size: cover;'><img class='photo_image' height='1' width='1' src='<%= @s3_direct_post.url %>/" + key + "'> </div>");

                // actual post to your controller

                  $.ajax({ type: "POST",
                  url: "/users/<%=@user.id%>/create_user_avatar",
                  data: { authenticity_token: '<%= form_authenticity_token %>', user_id: '<%= @user.id %>', key: key, url: url },
                  success: function(data) { },
                  error: function(data) {
                    var jqxHR = data.jqXHR;
                    var status = data.textStatus;
                    var error_message = data.errorThrown;
                    var key = $(data.jqXHR.responseXML).find("Key").text();
                    photo_errors.append("<div class='col-md-11 col-sm-11 col-xs-11 headline_three' style='margin-top:10px; margin-right: 10px;'> There was an error uploading the photo " + key + " Error message: " + error_message + " Please attempt to upload this photo again.</div>");
                  }
                }); //ajax call

               } // if-else

            } // done function

          });

        });
      });

   </script>  
<% end %>
harrisjb
  • 461
  • 3
  • 7