Image file is not viewing uploaded by s3

Question

I am uploading image files using s3. But whenever I try to download using the URL as well as from the s3 console will download the image file. But, it will not visible to the image viewer. It just shows an incompatible file type.

myS3Function.uploadFile(request.body.fileName, request.files.myFileData, "image_folder").then(filename => {
 //success
})

.

const AWS = require('aws-sdk');

const s3 = new AWS.S3({
 region: process.env.REGION
});

exports.uploadFile  = (filename, data, folderName) => {
return new Promise((resolve, reject) => {
    const params = {
        Bucket: process.env.AWS_S3_BUCKET, 
        Key: folderName+'/'+filename,
        Body: data.data,
        ACL:'public-read',
        ContentType: "image/jpeg"
    };
    s3.upload(params, function(s3Err, data) {
        if (s3Err) reject(s3Err)
        console.log(`File uploaded successfully at ${data.Location}`)
        resolve(`${data.Location}`)

    });
});
}

I uploaded files using postman now as form data. I can see text files uploaded using this code correctly. Then why do images have the issue? Also, images and pdf actual file size is increased a little bit.

Answer 1

As indicated in the different comments of your question, there are several things that can motivate your problem.

Please, be sure that you are providing the necessary configuration about the different content types that should be considered binary. The AWS documentation provides great detail about it; this related SO question can be valuable as well.

Due to the fact you are using the serverless framework, as indicated in the links 1 2 you cited, please provide the necessary configuration there as well:

provider:
  apiGateway:
    binaryMediaTypes:
      - 'multipart/form-data'

In any way, it seems that even with this configuration, you are still facing the problem. You told you were able to successfully upload text files, but your images get corrupted, increasing their size: as indicated in the comments, it seems a clear indication that in some place the information is being converted to a different encoding, from binary to text, something like that. In fact, according to your dependencies, this seems to be the actual problem ,as reported in this issue of the serverless-http library and especially, in this others 1 and 2 of the serverless-offline library.

I think the issue is only local and that it will probably work without further problems in AWS.

In any way, as you can see in first of the above mentioned issues, the one related to serverless-http, the library has the following code:

return Buffer.from(event.body, event.isBase64Encoded ? 'base64' : 'utf8'); 

So, as a workaround, submitting your information as base 64 encoded can solve the issue: it is not a straightforward task if you are using form submission in your HTML - see for instance this great example for some ideas - although it can do the trick if you interact directly with your API from code. The only necessary change is in your params variable:

const params = {
  Bucket: process.env.AWS_S3_BUCKET, 
  Key: folderName+'/'+filename,
  Body: Buffer.from(data.data, 'base64'),
  ContentEncoding: 'base64',
  ContentType: 'image/jpeg',
  ACL:'public-read',
};

Please, note the use of Buffer.from(..., 'base64') and the inclusion of ContentEncoding: 'base64'.

In any way, if the code works in AWS I think the way to go would be waiting for the serverless-offline issues resolution.

Answer 2

I'm not sure what the problem is, but it seems like serverless offline is having trouble processing the data coming in as multipart/form-data. The easiest solution would be to encode the file as base64 and send the payload as application/json.

To encode the file as base64, use this website: https://base64.guru/converter/encode/file. It is trivial to do this locally and programatically, but the website is cross-platform and should be good enough for testing.

Request

Payload:

{

    "fileName": "sampleFile.jpg",
    "myFileData": "R0lGODlhPQBEAPeoAJosM//AwO/AwHVYZ/z595kzAP/s7P+goOXMv8+fhw/v739/f+8PD98fH/8mJl+fn/9ZWb8/PzWlwv///6wWGbImAPgTEMImIN9gUFCEm/gDALULDN8PAD6atYdCTX9gUNKlj8wZAKUsAOzZz+UMAOsJAP/Z2ccMDA8PD/95eX5NWvsJCOVNQPtfX/8zM8+QePLl38MGBr8JCP+zs9myn/8GBqwpAP/GxgwJCPny78lzYLgjAJ8vAP9fX/+MjMUcAN8zM/9wcM8ZGcATEL+QePdZWf/29uc/P9cmJu9MTDImIN+/r7+/vz8/P8VNQGNugV8AAF9fX8swMNgTAFlDOICAgPNSUnNWSMQ5MBAQEJE3QPIGAM9AQMqGcG9vb6MhJsEdGM8vLx8fH98AANIWAMuQeL8fABkTEPPQ0OM5OSYdGFl5jo+Pj/+pqcsTE78wMFNGQLYmID4dGPvd3UBAQJmTkP+8vH9QUK+vr8ZWSHpzcJMmILdwcLOGcHRQUHxwcK9PT9DQ0O/v70w5MLypoG8wKOuwsP/g4P/Q0IcwKEswKMl8aJ9fX2xjdOtGRs/Pz+Dg4GImIP8gIH0sKEAwKKmTiKZ8aB/f39Wsl+LFt8dgUE9PT5x5aHBwcP+AgP+WltdgYMyZfyywz78AAAAAAAD///8AAP9mZv///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAAKgALAAAAAA9AEQAAAj/AFEJHEiwoMGDCBMqXMiwocAbBww4nEhxoYkUpzJGrMixogkfGUNqlNixJEIDB0SqHGmyJSojM1bKZOmyop0gM3Oe2liTISKMOoPy7GnwY9CjIYcSRYm0aVKSLmE6nfq05QycVLPuhDrxBlCtYJUqNAq2bNWEBj6ZXRuyxZyDRtqwnXvkhACDV+euTeJm1Ki7A73qNWtFiF+/gA95Gly2CJLDhwEHMOUAAuOpLYDEgBxZ4GRTlC1fDnpkM+fOqD6DDj1aZpITp0dtGCDhr+fVuCu3zlg49ijaokTZTo27uG7Gjn2P+hI8+PDPERoUB318bWbfAJ5sUNFcuGRTYUqV/3ogfXp1rWlMc6awJjiAAd2fm4ogXjz56aypOoIde4OE5u/F9x199dlXnnGiHZWEYbGpsAEA3QXYnHwEFliKAgswgJ8LPeiUXGwedCAKABACCN+EA1pYIIYaFlcDhytd51sGAJbo3onOpajiihlO92KHGaUXGwWjUBChjSPiWJuOO/LYIm4v1tXfE6J4gCSJEZ7YgRYUNrkji9P55sF/ogxw5ZkSqIDaZBV6aSGYq/lGZplndkckZ98xoICbTcIJGQAZcNmdmUc210hs35nCyJ58fgmIKX5RQGOZowxaZwYA+JaoKQwswGijBV4C6SiTUmpphMspJx9unX4KaimjDv9aaXOEBteBqmuuxgEHoLX6Kqx+yXqqBANsgCtit4FWQAEkrNbpq7HSOmtwag5w57GrmlJBASEU18ADjUYb3ADTinIttsgSB1oJFfA63bduimuqKB1keqwUhoCSK374wbujvOSu4QG6UvxBRydcpKsav++Ca6G8A6Pr1x2kVMyHwsVxUALDq/krnrhPSOzXG1lUTIoffqGR7Goi2MAxbv6O2kEG56I7CSlRsEFKFVyovDJoIRTg7sugNRDGqCJzJgcKE0ywc0ELm6KBCCJo8DIPFeCWNGcyqNFE06ToAfV0HBRgxsvLThHn1oddQMrXj5DyAQgjEHSAJMWZwS3HPxT/QMbabI/iBCliMLEJKX2EEkomBAUCxRi42VDADxyTYDVogV+wSChqmKxEKCDAYFDFj4OmwbY7bDGdBhtrnTQYOigeChUmc1K3QTnAUfEgGFgAWt88hKA6aCRIXhxnQ1yg3BCayK44EWdkUQcBByEQChFXfCB776aQsG0BIlQgQgE8qO26X1h8cEUep8ngRBnOy74E9QgRgEAC8SvOfQkh7FDBDmS43PmGoIiKUUEGkMEC/PJHgxw0xH74yx/3XnaYRJgMB8obxQW6kL9QYEJ0FIFgByfIL7/IQAlvQwEpnAC7DtLNJCKUoO/w45c44GwCXiAFB/OXAATQryUxdN4LfFiwgjCNYg+kYMIEFkCKDs6PKAIJouyGWMS1FSKJOMRB/BoIxYJIUXFUxNwoIkEKPAgCBZSQHQ1A2EWDfDEUVLyADj5AChSIQW6gu10bE/JG2VnCZGfo4R4d0sdQoBAHhPjhIB94v/wRoRKQWGRHgrhGSQJxCS+0pCZbEhAAOw==",
    "fileType": "image/jpeg",
    "productId": 12,
    "isDefault": false,
    "position": 2
}

product-image.js

const s3Functions = require('../services/s3Functions')
const { KEY_S3_PRODUCT_IMAGES_FOLDER } = require('../util/constants');
const { KEY_STATUS, KEY_DATA } = require('../util/constants');

exports.uploadProductImage = (body) => {
    return new Promise((resolve, reject) => {
        s3Functions.uploadFile(
            body.fileName,
            body.myFileData,
            body.fileType,
            KEY_S3_PRODUCT_IMAGES_FOLDER
        ).then(filename => {
            resolve({ [KEY_STATUS]: 1, [KEY_STATUS]: "Uploaded successfully", [KEY_DATA]: filename });
        }).catch(error => {
            reject({ [KEY_STATUS]: 0, [KEY_STATUS]: "Upload Failed", [KEY_DATA]: error });
        });
    })
}

s3Functions.js

const AWS = require('aws-sdk');

const s3 = new AWS.S3({
    params: { Bucket: 'rename-me', ACL:'public-read' },
});

exports.uploadFile = (
    filename,
    data,
    contentType,
    folderName
) => {
    console.log('Uploading')
    return new Promise((resolve, reject) => {
        const params = {
            Key: folderName+'/'+filename,
            Body: Buffer.from(data, 'base64'),
            ContentEncoding: 'base64',
            ContentType: contentType,
        };
        s3.upload(params, function(s3Err, data) {
            if (s3Err) {
                console.error(s3Err);
                reject(s3Err)
            }
            console.log(`File uploaded successfully at ${data.Location}`)
            resolve(`${data.Location}`)
        });
    });
}

Here's the codebase you provided on the other thread, with the included changes. I haven't cleaned it up, but just in case you run into any issues.

Answer 3

You need to send Body as a Buffer or ReadableStream, so you need to read the request.files.image as a Buffer before send it to s3 api. Also add content type to request headers

I'm not using your same stack but this is my working code

CURL testing command

curl -X 'PUT' \
 'http://localhost/api' \
 -H 'accept: _/_' \
 -H 'Content-Type: multipart/form-data' \
 -F 'image=@/Users/some/dir/userprofilesample1.jpg;type=image/jpeg'

S3 api call - I'm not specifying ContentType

.upload({
        Body: content, // Buffer | ReadableStream
        Bucket: bucketName,
        Key: path,
        ACL:  'public-read',
      })

https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html

Answer 4

Hi I would suggest you to use binary option while uploading the pic instead of Multi-part. Then try downloading it. That will solve your issue. See postman example snapshot here

If You want to use multipart for image upload on s3 then you'll have to create presigned url for each part then only you can use put to upload your file.

Here is a reference article where you could find multipart implementation

https://dev.to/traindex/multipart-upload-for-large-files-using-pre-signed-urls-aws-4hg4