C# - AWS EC2 Outgoing Traffic Does Not Initiate

Question

we are using Cloudinary and AWS S3 Bucket as CDN for our application.

At my local development machine, the Cloudinary.NET and AWS SDK works as expected and can upload files. However when I publish the application to the EC2 instance, it never even initiates the HTTP(s) connection. I can see that through WireShark and Microsoft Network Monitor tools. No TCP requests are made, and the code never reaches the lines after the network calls (the SDKs' methods).

To test things out, I even tried implement the API calls using C#'s HttpClient to no avail. Somehow the calls to the HTTP protocol from C# are not processed at all with no exception. It acts like an infinite timeout.

Since I get no errors at all, I have no idea what I am supposed to do.

NOTE The EC2 instance's Security Group allows ALL outgoing traffic by the way. And it also allows incoming traffic for Ephimeral ports (whatever that is).

Any directions are appreciated.

Here is the code snippet for HttpClient:

using (HttpClient c = new HttpClient())
{
    var fileBytes = new byte[model.FileStream.Length];

    _logger.LogInformation("Reading bytes from incoming file stream...");
    await model.FileStream.ReadAsync(fileBytes, 0, fileBytes.Length);
    model.FileStream.Close();
    _logger.LogInformation("Read bytes from incoming file stream...");

    MultipartFormDataContent form = new MultipartFormDataContent();

    // Also tried including an actual file as ByteArrayContent
    //form.Add(new ByteArrayContent(fileBytes, 0, fileBytes.Length), "file");
    form.Add(new StringContent("SOME_PUBLICLY_ACCESSABLE_URL"), "file");
    form.Add(new StringContent(_cloudinarySettings.ApiKey), "api_key");
    form.Add(new StringContent(timestamp.ToString()), "timestamp");
    form.Add(new StringContent(signature), "signature");

    HttpResponseMessage response = await c.PostAsync(
        $"https://api.cloudinary.com/v1_1/{_cloudinarySettings.Cloud}/image/upload", 
        form);

    _logger.LogInformation("RESPONSE: {0}", await response.Content.ReadAsStringAsync());

    return null;
}

For Cloudinary:

var uploadParams = new ImageUploadParams()
{
    File = new FileDescription(fileName, model.FileStream),
    PublicId = publicId,
    Overwrite = true,
    // TODO: 
    NotificationUrl = model.CallbackUrl
};

_logger.LogInformation("Trying to upload to CDN");

var uploadResult = await _cloudinary.UploadAsync(uploadParams);

_logger.LogInformation("Uploaded image to CDN");

var url = _cloudinary.Api.UrlImgUp
    .Format(uploadResult.Format)
    .Transform(new Transformation().FetchFormat("auto"))
    .BuildUrl(uploadResult.PublicId);

AWS SDK:

var fileTransferUtility =
    new TransferUtility(_amazonS3Client);

var objectId = $"{folder}/{fileName}";

var fileTransferUtilityRequest = new TransferUtilityUploadRequest
{
    BucketName = _awsS3Settings.BucketName,
    InputStream = model.FileStream,
    StorageClass = S3StorageClass.Standard,
    PartSize = 6291456, // 6 MB.
    Key = objectId,
    CannedACL = S3CannedACL.PublicRead
};
fileTransferUtilityRequest.Metadata.Add("X-FileExtension", Path.GetExtension(model.FileName));
fileTransferUtilityRequest.Metadata.Add("X-OriginalFileName", model.FileName);
fileTransferUtilityRequest.Metadata.Add("X-GeneratedFileId", fileId.ToString());
fileTransferUtilityRequest.Metadata.Add("X-GeneratedFileName", fileName);

await fileTransferUtility.UploadAsync(fileTransferUtilityRequest);

BONUS

I even included a simple GET call to google.com with no success again at the

using (HttpClient c = new HttpClient())
{
    _logger.LogInformation("Sending request to google...");

    c.Timeout = TimeSpan.FromSeconds(2);
    HttpResponseMessage response = await c.GetAsync(
        $"https://google.com");

    _logger.LogInformation("RESPONSE: {0}", await response.Content.ReadAsStringAsync());

    return null;
}

score 0 · Answer 1 · answered Feb 03 '22 at 11:20

This is weird but it seems like the default HttpClient somehow uses system level default proxy or something?

The answer to the question Web request from HttpClient stuck is kind of correct. The solution I came up with was forking the CloudinaryDotNet repository, inject a new HttpClientHandler with Proxy explicitly set to null and UseProxy explicitly set to false. This behaviour is not supported because the library internally creates a new HttpClient once at ApiShared.Proxy.cs and it internally decides based on target framework whether to use the HttpClientHandler or not.

For reference, this is the code change that makes it work:

ApiShared.Proxy.cs Line 17

Remove

        public HttpClient Client = new HttpClient();

Add

        public HttpClient Client = new HttpClient(new HttpClientHandler
        {
            UseProxy = false,
            Proxy = null
        });

Obviously this will not work for everyone and is not an ideal solution. The ideal solution would probably involve digging deeper on how EC2 uses default proxies, maybe somehow including the proxy options the SDKs. But I'm posting anyway in case someone else has a similar problem.

C# - AWS EC2 Outgoing Traffic Does Not Initiate

1 Answers1

Remove

Add