How do I upload to Amazon S3 using .NET HttpClient WITHOUT using their SDK

Question

How do I upload a file to a bucket on Amazon S3 just using .NET's HttpClient (or WebClient)? It has to be done using "PUT".

I can upload using Amazon's AWS SDK, but I would like to know how to do it without it. I've looked at their documentation for hours and still don't get it.

Any help would be greatly appreciated.

My code using their SDK is:

public static void TestPutObjectAsync() {
        AmazonS3Client client = new AmazonS3Client();
        client.AfterResponseEvent += new ResponseEventHandler(callback);

        PutObjectRequest request = new PutObjectRequest {
            BucketName = BUCKET_NAME,
            Key = "Item1",
            FilePath = FILE_PATH,
        };

        client.PutObjectAsync(request);
}

public static event EventHandler UploadComplete;

public static void callback(object sender, ResponseEventArgs e) {

        if (UploadComplete != null) {
            UploadComplete(sender, e);
        }
}

Answer 1

I've come across the question when trying to deploy file uploading in a Xamarin .net core cross platform client where the presigned post was generated server-side using boto3.

The requirement is to not use the SDK client-side because while this question is about AWS S3, many other services (Wasabi & Digital Ocean to name a couple) use the same API and the configuration of the SDK client-side for a different service can be too much to manage - the server should handle generating the presigned POST for various service endpoints and the client shouldn't have to mirror that endpoint. This way, our server can easily change endpoints and the client doesn't require an update.

1: Create supporting models for the presigned post. This stems from https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3.html#generating-presigned-posts

public class PresignedPost
{
    public string url { get; set; }
    public Fields fields { get; set; }
}
public class Fields
{
    public string AWSAccessKeyId { get; set; }
    public string acl { get; set; }
    public string key { get; set; }
    public string policy { get; set; }
    public string signature { get; set; }
}

2: Load the file. In our case we read into memory via

string data = File.ReadAllText(filepath)

Yes, this may create issues with large files in memory, but currently the limitation is not addressed in our requirements.

3: Create the Upload method using an HttpClient from System.Net.Http

public static async Task<bool> Upload(PresignedPost presignedPost, string data)
{            
    using (HttpClient client = new HttpClient())
    {
        try
        {
            MultipartFormDataContent formData = new MultipartFormDataContent
            {
                { new StringContent(presignedPost.fields.acl), "acl" },
                { new StringContent(presignedPost.fields.AWSAccessKeyId), "AWSAccessKeyId" },
                { new StringContent(presignedPost.fields.key), "key" },
                { new StringContent(presignedPost.fields.policy), "policy" },
                { new StringContent(presignedPost.fields.signature), "signature" },
                { new StringContent(data), "file" },
            };
            var response = await client.PostAsync(presignedPost.url, formData);
            //if response is 204 no content it's successful
            if (response.StatusCode == HttpStatusCode.NoContent)
            {
                return true;
            }
        }
        catch (Exception ex)
        {
            //do something with exception
        }
    }
    return false;
}

Comments:

1: It also also possible to load the file as a byte array and then using a memory stream in the formData. This may be helpful when uploading larger files however, our stack encrypts the file first and returns a string so we did not choose that method in this implementation. ref: C# HttpClient 4.5 multipart/form-data upload Or simply use a byte array instead of a string ref: send byte array by HTTP POST in store app

2: You may notice that we assume a 204 Response code means a successful upload. While this is obviously dangerous, we couldn't find any documentation on why this occurs - perhaps its the same reasoning why AWS S3 Object DELETE also returns a 204. If you're concerned about this, you could tack on an API call to check if the file exists.

3: Yes, there are arguments over using "using" to dispose of HttpClients and you may chose based on your use-case ref: https://aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/

4: Return a bool or not, again please choose based on your own practices and requirements.

Lastly, I hope this helps anyone facing this issue as an example was not readily available from the AWS S3 docs.

Answer 2

Not sure why you don't want to use the SDK (thats what its there for), but even if you don't want to use the whole AWSSDK, you can still use parts of it - or at least see how they do it under the covers.

Its all open source here:

https://github.com/aws/aws-sdk-net/tree/master/AWSSDK_DotNet45/Amazon.S3

Ultimately, the answer you are looking for is in there - but it may be more work than its worth pulling it out.