Rails upload to AWS creating .zip.cpgz file loop

Question

I'm running into a weird situation where some files, specifically ZIP format, when uploaded to AWS within my Rails app are corrupted/converted. When downloaded and decompressed they turn into a CPGZ format, which decompresses back into a ZIP, and infinitely does this.

I haven't noticed a pattern that causes this, so it's seemingly sporadic, and can confirm that the files are not corrupt before upload. The only other issue/topic I've found on this relates to PHP, and seems to be different circumstances.

I am using AWS SDK for Ruby v1 (not v2 because of my Rails version) and jQuery-File-Upload. Since some of the files are large, I am using chunked uploads.

In my controller, the presigned POST URL is created like this:

S3_BUCKET.presigned_post(key: "uploads/#{SecureRandom.uuid}-${filename}", success_action_status: '201')

And jQuery File Upload is set up like so (some parts removed for brevity):

this.$el.fileupload({
  fileInput: this.uploadField, // this is an <input type="file">
  url: this.awsURL, // https://BUCKET.s3.amazonaws.com/
  formData: JSON.parse(this.awsData), // {"AWSAccessKeyId":"...","key":"uploads/1234-${filename}","policy":"...","signature":"...","success_action_status":"201"}
  type: 'POST',
  autoUpload: true,
  paramName: 'file',
  dataType: 'XML',
  replaceFileInput: false,
  maxChunkSize: 1000000,
  add: function(event, data) {
    var file = data.files[0];
    var fileType = file.type;

    // Check file type
    if (~'ai sketch psd jpg jpeg png zip ttf woff eot gif'.indexOf(fileType.toLowerCase())) {
      return alert('Sorry, that file type is not supported');
    };

    data.submit();
  },
  progress: function(event, data) {
    // Display progress
  },
  done: function(event, data) {
    var file = data.files[0];
    var fileName = file.name.replace(/ /g,"_");
    var item = _this.uploadedItems[fileName];
    var key = $(data.jqXHR.responseXML).find("Key").text();
    // awsHost = BUCKET.s3.amazonaws.com
    var url = '//' + _this.awsHost + '/' + key;

    // Set form values using above info
  },
  fail: function(event, data) {
    // Alert failure
  }
});

Has anyone experienced this? It's very frustrating.

Have you tried "uploading" to a temp folder on your local disk? If so, does the same corruption occur? — Anthony E, Mar 30 '16 at 05:04

score 2 · Accepted Answer · answered Apr 02 '16 at 11:15

2

Set content-type to application/zip when you sending the request.

SEE https://github.com/aws/aws-sdk-ruby/blob/aws-sdk-v1/lib/aws/s3/presigned_post.rb

answered Apr 02 '16 at 11:15

Shishir

780
4
9

Thanks for the suggestion! I changed my POST URL to `S3_BUCKET.presigned_post(key: "uploads/#{SecureRandom.uuid}-${filename}", content_type: 'application/zip', success_action_status: '201')` but unfortunately I still get the CPGZ file upon upload. – Jody Heavener Apr 02 '16 at 20:36
1

Does it work if you are not chunking? or do u face the same issue. trying to isolate what is causing the data corruption and when? – Shishir Apr 08 '16 at 21:48
your suggestion ended up being the fix here! I did not have chunking set up properly. Feel free to create an official answer and I will accept it. – Jody Heavener Jun 21 '16 at 16:51

Rails upload to AWS creating .zip.cpgz file loop

1 Answers1