0

I'm looking for a secure way to directly upload large files to S3 (for performance issue).

After few hours of research, I've come to the (maybe incorrect) conclusion that I should use "Browser-Based Uploads Using POST". As referenced in this thread: Amazon S3 direct file upload from client browser - private key disclosure

Before trying this directly, I thought about making a cURL POC with direct upload and signature computation. I've failed to make it work, and haven't found a successful POC on the web using Authentification Version 4 manual settings.

My signature is OK. The only issue is that Amazon is double SHA256ing my file content and is thus not validation my x-amz-content-sha-256 header.

  • lower(SHA256(e8379a31b13fb9423928fe28dd41a5e3204a52072634503c31e8b3ea42605b46)) = 4fa84cd7d18e0d33dbd62d0492eca4a159e122391ae0a3e636bd3cf527680c87

I'm not sure of understanding what should I put in my cURL and canonical request payload (and the linked content-length value) and the x-amz-content-sha-256 header. Should it all be the same values ? If yes, then the Amazon doc specifies it should all be encrypted to SHA256, so I've no idea why Amazon reSHA256 my already SHA256 payload...

Error:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>XAmzContentSHA256Mismatch</Code>
<Message>The provided 'x-amz-content-sha256' header does not match what was computed.</Message><ClientComputedContentSHA256>e8379a31b13fb9423928fe28dd41a5e3204a52072634503c31e8b3ea42605b46</ClientComputedContentSHA256><S3ComputedContentSHA256>4fa84cd7d18e0d33dbd62d0492eca4a159e122391ae0a3e636bd3cf527680c87</S3ComputedContentSHA256>
<RequestId>419A185269B0F891</RequestId><HostId>QHWxK0Mzz6AfG44ypXBti3W0tYx1xkG9lZGqc2kUKyMF9STwP18M3racio0k06aH5+1ok/Irdn8=</HostId>
</Error>

cURL command:

curl 
-v https://??.s3.amazonaws.com/recordtest/test.jpg        
-H "Authorization: AWS4-HMAC-SHA256 Credential=??/20170228/eu-west-1/s3/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=43750caa762314eb70aace1f7f8ae34633b93352aa25646433ef21e48dd79429"        -H "Content-Length: 64"        
-H "Content-Type: application/octet-stream"        
-H "x-amz-content-sha256: e8379a31b13fb9423928fe28dd41a5e3204a52072634503c31e8b3ea42605b46"        
-H "x-amz-date: 20170228T111828Z"        
-d "e8379a31b13fb9423928fe28dd41a5e3204a52072634503c31e8b3ea42605b46"        
-X PUT

Generated canonical request:

PUT
/recordtest/test.jpg

content-length:64
content-type:application/octet-stream
host:??.s3.amazonaws.com
x-amz-content-sha256:e8379a31b13fb9423928fe28dd41a5e3204a52072634503c31e8b3ea42605b46
x-amz-date:20170228T111200Z

content-length;content-type;host;x-amz-content-sha256;x-amz-date
e8379a31b13fb9423928fe28dd41a5e3204a52072634503c31e8b3ea42605b46
Community
  • 1
  • 1
Azzip
  • 122
  • 10
  • 1
    Using `-d "e8379a31b13fb9423928fe28dd41a5e3204a52072634503c31e8b3ea42605b46"`, you are telling curl that this string of digits is the request body -- this is the content of the file (object) that you want to upload. That is almost certainly not what you intend to do, but explains exactly why the sha256 error occurs. The canonical request is a canonical *representation* of the request. You wouldn't actually transmit that last line (e8379a...) in the request you send. I believe you actually want `--data-binary '@local_file_name'`. – Michael - sqlbot Feb 28 '17 at 12:41
  • 1
    Note also for reference that SHA256 is not encryption, it's a cryptographic hash, which is a vastly different thing than encryption. – Michael - sqlbot Feb 28 '17 at 12:43
  • I've replaced my cURL command and added --data-binary "@./img/$img_file" and I've now got this error: *GnuTLS recv error (-54): Error in the pull function. – Azzip Feb 28 '17 at 12:58
  • And now : curl : (55) Failed sending data to the peer – Azzip Feb 28 '17 at 13:05
  • And if I retry enough times, I occasionaly get the samer error as before (the provided 'x-amz-content-sha256' header does not match what was computed)... Should I modify something else than the binary payload of the cURL request ? Like the content-length ? – Azzip Feb 28 '17 at 13:42
  • 1
    Yes, the `Content-Length` header in the request and in the signing process needs to be set to the size of the file to be uploaded, in bytes. – Michael - sqlbot Feb 28 '17 at 14:28
  • Thank you a lot. Setting both Content-Length header to the byte length of the non-hashed file made it work. I will try a "real" XmlHttpRequest POST now... – Azzip Feb 28 '17 at 15:21

0 Answers0