1

Uploading files via FilesResource.Update or FilesResource.Create is regularly emitting IOExceptions with message "System.IO.IOException: Unable to read data from the transport connection: The connection was closed." Is there any chance I can prevent these IOExceptions from happening?

The uploads seem to succeed, eventually, probably because this is happening within the ResumableUpload facility in the API. But (maybe anecdotally) it's causing not-insignificant performance issues; at times the queue of work being processed by our upload processes slows to almost a standstill, with these errors being logged incessantly.

Utility for uploading streams to Drive:

public async Task UploadFile(Stream uploadStream, string filename, string parentFolderId, string mimeType="", CancellationToken token = default(CancellationToken))
{
    try
    {
        // File information for where this content will end up - name, folder, mime type
        var fileMetaData = new File();
        fileMetaData.Name = filename;
        fileMetaData.MimeType = mimeType;

        // check if file by this name exists with same parent and not trashed
        var results = await ListFiles($"mimeType!='{FolderMimeType}' and name='{filename}' and '{parentFolderId}' in parents and trashed=false", token);
        if (results.Count > 0)
        {
            log.Debug($"Overwrite existing fileid {results[0].Id} \"{results[0].Name}\" in parent {parentFolderId}");
            var request = driveService.Files.Update(fileMetaData, results[0].Id, uploadStream, fileMetaData.MimeType);
            request.Fields = "id, name";
            request.ProgressChanged += Upload_ProgressChanged;
            request.ResponseReceived += Upload_ResponseReceived;
            request.SupportsAllDrives = true; // required in order to be able to upload to shared drive

            await request.UploadAsync(token);
        }
        else
        {
            log.Debug($"No existing file found, uploading {filename} to parent {parentFolderId}");
            fileMetaData.Parents = new List<string> { parentFolderId };

            // send the stream to the destination file and dispose when finished
            var request = driveService.Files.Create(fileMetaData, uploadStream, fileMetaData.MimeType);
            request.Fields = "id, name";
            request.ProgressChanged += Upload_ProgressChanged;
            request.ResponseReceived += Upload_ResponseReceived;
            request.SupportsAllDrives = true; // required in order to be able to upload to shared drive

            await request.UploadAsync(token);
        }
    }
    catch (Exception e)
    {
        throw e;
    }
    finally
    {
        uploadStream.Dispose();
    }
}

These exceptions are caught in the "ProgressChanged" event, where the upload's status gets marked as Failed even though it does seem to be retried by the ResumableUpload part of the API:

static void Upload_ProgressChanged(IUploadProgress progress)
{
    log.Debug($"Status: {progress.Status}, {BytesToString(progress.BytesSent)} sent");
    if (progress.Exception != null && progress.Status == Google.Apis.Upload.UploadStatus.Failed)
    {
        log.Error($"Upload failed: {progress.Exception.Message}", progress.Exception);
    }
}

Log w/ stack trace:

2022-05-23 17:25:19,365|ERROR|62|Upload failed: Unable to read data from the transport connection: The connection was closed.

System.IO.IOException: Unable to read data from the transport connection: The connection was closed.
   at System.Net.ConnectStream.EndRead(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncTrimPromise`1.Complete(TInstance thisRef, Func`3 endMethod, IAsyncResult asyncResult, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Google.Apis.Upload.ResumableUpload.UploadBuffer.<PopulateFromStreamAsync>d__11.MoveNext() in C:\Apiary\2021-09-08.15-52-39\Src\Support\Google.Apis\Upload\ResumableUpload.cs:line 844
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Google.Apis.Upload.ResumableUpload.<PrepareBufferUnknownSizeAsync>d__81.MoveNext() in C:\Apiary\2021-09-08.15-52-39\Src\Support\Google.Apis\Upload\ResumableUpload.cs:line 720
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Google.Apis.Upload.ResumableUpload.<SendNextChunkAsync>d__77.MoveNext() in C:\Apiary\2021-09-08.15-52-39\Src\Support\Google.Apis\Upload\ResumableUpload.cs:line 630
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Google.Apis.Upload.ResumableUpload.<UploadCoreAsync>d__74.MoveNext() in C:\Apiary\2021-09-08.15-52-39\Src\Support\Google.Apis\Upload\ResumableUpload.cs:line 572
Nick Eby
  • 21
  • 2
  • Does this answer your question? [Unable to read data from the transport connection : An existing connection was forcibly closed by the remote host](https://stackoverflow.com/questions/5420656/unable-to-read-data-from-the-transport-connection-an-existing-connection-was-f) – PatrickdC May 24 '22 at 00:26
  • Thanks for the link. Unfortunately I don't think anything here is directly applicable because I can't assert control over the client that's embedded inside the Google Drive SDK for .NET. I could try to roll my own client....... but, I'm not going to. – Nick Eby May 24 '22 at 04:40
  • have you checked the [media_upload](https://developers.google.com/api-client-library/dotnet/guide/media_upload) docs? – Linda Lawton - DaImTo Jun 28 '22 at 08:16

0 Answers0