Azure server throwing exception after certain numbers of API call

Question

I have created an API (MVC) and Hosted it in Azure server. The API is receiving an 1 Image and 1 Text at a time. It is working properly for the single call or first call but after certain numbers of API call getting below error.

After installing Microsoft.Azure.DocumentDB :

Error:

=== Pre-bind state information === LOG: DisplayName = Microsoft.Azure.Documents.Client, Version=1.11.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35 (Fully-specified) LOG: Appbase = file:///D:/Project/002 MVC API/Cherish/API/Cherish.Api/ LOG: Initial PrivatePath = D:\Project\002 MVC API\Cherish\API\Cherish.Api\bin Calling assembly : Cherish.Domain, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null. === LOG: This bind starts in default load context. LOG: Using application configuration file: D:\Project\002 MVC API\Cherish\API\Cherish.Api\web.config LOG: Using host configuration file: C:\Users\Yudiz\Documents\IISExpress\config\aspnet.config LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config. LOG: Post-policy reference: Microsoft.Azure.Documents.Client, Version=1.11.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35 LOG: Attempting download of new URL file:///C:/Users/Yudiz/AppData/Local/Temp/Temporary ASP.NET Files/vs/7b6de7f7/4f48effb/Microsoft.Azure.Documents.Client.DLL. LOG: Attempting download of new URL file:///C:/Users/Yudiz/AppData/Local/Temp/Temporary ASP.NET Files/vs/7b6de7f7/4f48effb/Microsoft.Azure.Documents.Client/Microsoft.Azure.Documents.Client.DLL. LOG: Attempting download of new URL file:///D:/Project/002 MVC API/Cherish/API/Cherish.Api/bin/Microsoft.Azure.Documents.Client.DLL. WRN: Comparing the assembly name resulted in the mismatch: Major Version ERR: Failed to complete setup of assembly (hr = 0x80131040). Probing terminated.

Here is my code snippet:

var blobHelper = new BlobHelper();
var response = new UploadMediaResponse();
for (var i = 0; i<provider.Files.Count; i++)
{
    Guid itemId;
    var fileData = await provider.Files[i].ReadAsByteArrayAsync();
    if (mediaType == MediaType.Thumbnail)
    {
        // MediaId is mandatory for Audio & Video media
        if (String.IsNullOrWhiteSpace(provider.FormData["mediaId"]))
        // ReSharper disable once NotResolvedInText
            return ErrorResponse(new ArgumentNullException("mediaId", "MediaId was not provided"));
        itemId = new Guid(provider.FormData["mediaId"]);
        await blobHelper.Upload(mediaType, accountId, itemId, fileData, timelineEvent.ChildId);
        var mediaItem = timelineEvent.Media.Single(m => m.Id == itemId);
        mediaItem.HasThumbnail = true;
        mediaItem.LastUpdated = DateTime.UtcNow;
        await timelineEventRepository.UpdateAsync(timelineEvent, entityDoc.SelfLink);
    }
    else
    {
        itemId = await blobHelper.Upload(mediaType, accountId, fileData, timelineEvent.ChildId);
        var mediaItem = new TimelineMediaItem
        {
            Id = itemId,
            Available = true,
            FileSize = fileData.Length,
            Removed = false,
            Type = mediaType.ToUpper(),
            TaggedChildren = new List<TaggedChild>(),
            Created = DateTime.UtcNow,
            LastUpdated = DateTime.UtcNow
        };
        if (fileLengthsRequired)
            mediaItem.Length = fileLengths[i];
        timelineEvent.Media.Add(mediaItem);
        await timelineEventRepository.UpdateAsync(timelineEvent, entityDoc.SelfLink);
    }
    response.MediaIds.Add(itemId);
}

Declaration for DocumentClient:

public DocumentRepository()
{
    Client = new DocumentClient(new Uri(ConfigurationManager.AppSettings["DocumentDbEndpointUrl"]), ConfigurationManager.AppSettings["DocumentDbAuthKey"]);
    DatabaseName = ConfigurationManager.AppSettings["DocumentDbDatabaseName"];
    PageSize = Int16.Parse(ConfigurationManager.AppSettings["DefaultPageSize"]);
    var _database = ReadOrCreateDatabase();
    var collection = InitialiseCollection(_database.SelfLink, EntityName);
    DocumentsLink = collection.DocumentsLink;
    SelfLink = collection.SelfLink;
}

Here is my error:

Error String :=> Optional({"Message":"An error has occurred.","ExceptionMessage":"One or more errors occurred.","ExceptionType":"System.AggregateException","StackTrace":" at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)\r\n at System.Threading.Tasks.Task1.GetResultCore(Boolean
waitCompletionNotification)\r\n at
System.Threading.Tasks.Task1.get_Result()\r\n at Microsoft.Azure.Documents.Linq.DocumentQuery1.GetEnumerator()\r\n

at System.Linq.Buffer1..ctor(IEnumerable1 source)\r\n at
System.Linq.Enumerable.ToArray[TSource](IEnumerable1 source)\r\n at Cherish.Domain.Repositories.Implementation.DocumentRepository1.ReadOrCreateDatabase()\r\n at
Cherish.Domain.Repositories.Implementation.DocumentRepository1..ctor()\r\n at Cherish.Api.Helpers.BlobHelper.AccountHasEnoughStorageCapacity(Guid accountId, Int32 fileSize)\r\n at Cherish.Api.Helpers.BlobHelper.d__4.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Cherish.Api.Helpers.BlobHelper.d__3.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)\r\n at Cherish.Api.Controllers.TimelineController.d__15.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Cherish.Api.Controllers.TimelineController.d__11.MoveNext()","InnerException":{"Message":"An error has occurred.","ExceptionMessage":"An error occurred while sending the request.","ExceptionType":"System.Net.Http.HttpRequestException","StackTrace":" at Microsoft.Azure.Documents.BackoffRetryUtility1.d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown
---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task
task)\r\n at
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at
Microsoft.Azure.Documents.Linq.DocumentQueryExecutionContext.d__7.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown
---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task
task)\r\n at
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at
Microsoft.Azure.Documents.Linq.DocumentQuery1.d__10.MoveNext()","InnerException":{"Message":"An error has occurred.","ExceptionMessage":"Unable to connect to the remote server","ExceptionType":"System.Net.WebException","StackTrace":" at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context)\r\n at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)","InnerException":{"Message":"An error has occurred.","ExceptionMessage":"An attempt was made to access a socket in a way forbidden by its access permissions","ExceptionType":"System.Net.Sockets.SocketException","StackTrace":" at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)\r\n at System.Net.Sockets.Socket.InternalBind(EndPoint localEP)\r\n at System.Net.Sockets.Socket.BeginConnectEx(EndPoint remoteEP, Boolean flowContext, AsyncCallback callback, Object state)\r\n at System.Net.Sockets.Socket.UnsafeBeginConnect(EndPoint remoteEP, AsyncCallback callback, Object state)\r\n at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)"}}}})

Thank You in advance.

Are you using Azure DocumentDB? I found a issue about the similar error, you could refer to [it](https://social.msdn.microsoft.com/Forums/en-US/0ddee231-d8f3-451f-8c94-954205ba02fc/azure-documentdb-error-unable-to-connect-to-the-remote-serveran-attempt-was-made-to-access-a-socket?forum=AzureDocumentDB). Would you please provide your core code snippet for us to locate this issue? — Bruce Chen, Mar 08 '17 at 02:19
@Bruce-MSFT Yes, I am using DocumentDB and I have updated the question with the code. — Chandresh Khambhayata, Mar 08 '17 at 05:00
@Bruce-MSFT : did you get the chance to check the above question please ? — Chandresh Khambhayata, Mar 10 '17 at 04:53
From the stacktrace, I noticed that the error was thrown at `DocumentRepository.ReadOrCreateDatabase()`. Based on your code, what does the variable `timelineEventRepository` represent for, the `DocumentRepository` class or something else? Have you tried to implement the singleton (e.g. creating a static instance of DocumentClient and reusing that static instance) for `DocumentClient` within the `DocumentRepository` class? — Bruce Chen, Mar 10 '17 at 09:21
`timelineEventRepository` is `DocumentRepository`. `singleton` you mean I have to define `DocumentClient` outside the `constructor` as a `Static Member`? Please correct me if I am wrong. — Chandresh Khambhayata, Mar 10 '17 at 11:09

score 1 · Accepted Answer · answered Mar 13 '17 at 03:20

1

According to your issue, I assumed that there be connection issues between your client and documentdb endpoint after certain numbers of API call. You could try to create a static instance of DocumentClient and add the retry policy to your DocumentRepository class as follows:

public class DocumentRepository
{
    private static readonly DocumentClient _client;
    static DocumentRepository()
    {
        var connectionPolicy = new ConnectionPolicy
        {
            RetryOptions = new RetryOptions()
            {
                MaxRetryAttemptsOnThrottledRequests = 5,
                MaxRetryWaitTimeInSeconds = 5
            }
        };
        _client = new DocumentClient(
            new Uri(ConfigurationManager.AppSettings["DocumentDbEndpointUrl"]),
            ConfigurationManager.AppSettings["DocumentDbAuthKey"],
            connectionPolicy);
    }

    public DocumentRepository()
    {
        DatabaseName = ConfigurationManager.AppSettings["DocumentDbDatabaseName"];
        PageSize = Int16.Parse(ConfigurationManager.AppSettings["DefaultPageSize"]);
        var _database = ReadOrCreateDatabase();
        var collection = InitialiseCollection(_database.SelfLink, EntityName);
        DocumentsLink = collection.DocumentsLink;
        SelfLink = collection.SelfLink;
    }

    public async Task UpdateAsync(TimelineEvent timelineEvent, string selfLink)
    {
        //TODO:
        await _client.UpsertDocumentAsync("{documentCollectionUri}", timelineEvent);
    }
}

Additionally, there is a official blog talking about performance tips for Azure DocumentDB, you could refer to it.

answered Mar 13 '17 at 03:20

Bruce Chen

18,207
2
21
35

I am getting error while using `RetryOptions` : `'ConnectionPolicy' does not contain a definition for 'RetryOptions'` – Chandresh Khambhayata Mar 14 '17 at 05:17
I used [Microsoft Azure DocumentDB Client Library 1.11.1](https://www.nuget.org/packages/Microsoft.Azure.DocumentDB) in my development, you could try to check the version of your client library. Any concerns, feel free to let me know. – Bruce Chen Mar 14 '17 at 05:28
I am using `Microsoft.Azure.Documents.Client` – Chandresh Khambhayata Mar 14 '17 at 05:33
I have added `Microsoft.Azure.DocumentDB` and the error is gone now. now I am testing the real issue. – Chandresh Khambhayata Mar 14 '17 at 05:36
Should I dispose `DocumentClient` ? – Chandresh Khambhayata Mar 14 '17 at 05:36
It is recommended to use a single instance of `DocumentClient` per AppDomain for the lifetime of your application. You could firstly check whether this issue could be solved. – Bruce Chen Mar 14 '17 at 05:42
I assumed that since you have updated the version of `Microsoft.Azure.DocumentDB`, there my be old version reside in your other projects references. You could refer to this similar [issue](http://stackoverflow.com/questions/4187907/net-picking-wrong-referenced-assembly-version). – Bruce Chen Mar 14 '17 at 06:22
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/137978/discussion-between-chandresh-khambhayata-and-bruce-msft). – Chandresh Khambhayata Mar 14 '17 at 06:27

score 1 · Answer 2 · edited Apr 23 '18 at 15:44

1

The accepted answer is actually not the solution to the issue above. Adding the retry info does hide some of the symptoms but it doesn't prevent it from happening again in the future.

If the DocumentClient is not properly defined AS A Singleton then its possible the application will run out of TCP connections and that's when you get the socket error.

Please refer to this link on MSDN for a little more information.

Thanks and I hope this helps others.

edited Apr 23 '18 at 15:44

iam.Carrot

4,976
2
24
71

answered Apr 23 '18 at 15:11

Kenneth Garza

1,886
14
12

This should be a comment. – bfontaine Apr 23 '18 at 15:21

Azure server throwing exception after certain numbers of API call

2 Answers2