IOS NSURLSession Background Uploading how to upload Large Files in Background?

Question

I want to Upload the Large Files (Videos) of 2-3 GB to Server in background with the following requirements

First Method

Uploading should resume if the internet Connection lost and reconnected
Uploading should continue even application is in background
Uploading should resume if user kills the application and comes back

for the Above features what I have implemented

User select the File
Split the File into Chunks Of 1MB and save all the Chunks on the Disk as File
Create the Upload Task Against Each Chunk File and add the File in Background Session

Above Method works but fails in Some Case

If the File Is Larger than 1GB Creating Chunks and Writing Chunks on Disk throw Memory Exception
If I want to Upload the File of 1GB I need extra 1 GB Space to Create Chunks

Second Method

Upload the Original File without creating chunks, in this case, I am not able to resume uploading if network connectivity lost or User Kill the application

My question is What is the best way to upload the Large files in Background Keeping all these points in Mind

I know there are some questions of this type already asked but none of them answer my question

I have spent lot of time Implementing this but can not implement it successfully please help me or give some suggestion what is the best way to complete the above points

Update

I am Using the Following Code to Create Chunks Code is in Xamarin.IOS but i am Ok if some one Provide explantion in Objective C or Swift

public static void SplitFileInChunks( UploadFileInfo UploadFile )
{
        int i = -1;

        long chunkSize = UploadHelper.chunkSize;
        nuint dataLength = (System.nuint)chunkSize; 

        //var documents = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments);
        string directoryPath = UploadHelper.UploadsDirectory;

        int chunkCount = 0;
        NSFileHandle fileHandleRead = NSFileHandle.OpenRead(UploadFile.FilePath);

        fileHandleRead.ReadInBackground();
        //fileHandleRead.WaitForDataInBackground();

        if (fileHandleRead == null)
            return;

        do
        {
            i++;
            ulong index = (ulong)(i * chunkSize);

            var filePath = Path.Combine(directoryPath, UploadFile.ContentGuide + "" + i.ToString());
            //fileHandleRead.SeekToFileOffset(index);

            NSData data = fileHandleRead.ReadDataOfLength(dataLength );
            Console.WriteLine(UploadFile.FileStatus);

            if (data.Length <= 0)
                continue;

            NSFileManager.DefaultManager.CreateFile(filePath, data, attr: null);


            NSError error;
            //data.Save(filePath, true, out error);

            chunkCount++;

            Console.WriteLine("Data Lenght" + data.Length);
            data.Dispose();
            Console.WriteLine("Chunk " + i);
        }

        while ( i * chunkSize <= UploadFile.Size && UploadFile.FileStatus != UploadFileStatus.Aborted );

        fileHandleRead.CloseFile();
        fileHandleRead.Dispose();

        Console.WriteLine("All Files Written sucessuflly");
        UploadFile.TotalChunksCount = chunkCount;

    }

yes, I am using background session but to save chunks on the disk I need extra space because NSURLsession in the background only works With NSURL.FromFileName as far as I understand — Fahad Rehman, Apr 24 '17 at 14:50
so is there any other way to overcome this, like upload the Original file and also support resuming — Fahad Rehman, Apr 24 '17 at 14:53
I have tried to add this in Auto `ReleasePool` block but no luck — Fahad Rehman, Apr 24 '17 at 14:55
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/142512/discussion-between-rob-and-fahad-rehman). — Rob, Apr 24 '17 at 14:55

score 1 · Accepted Answer · answered Apr 24 '17 at 23:03

1

That will certainly work, but if you're in control over the software on the other end, you can do better:

On the server side:
- Provide an upload start endpoint (URL) that merely provides a unique ID.
- Provide an upload data endpoint that takes the unique ID, a POST body, and an optional starting byte offset, and writes the data to a temporary file on the server.
- Provide an upload status endpoint that takes the unique ID and returns the amount of data it has stored on disk so far.
- Provide an upload finished endpoint.
On the client side:
- Call the start endpoint and get an ID for the upload.
- Call the upload data endpoint and start sending data.
- On failure, call the upload status endpoint to find out how much data the server actually got.
- Then call the data endpoint and start sending data from that offset, telling the server where you're starting. (On the server, always start writing at that offset into the file even if the length has increased since then, just to be safe.)
- Upon completion, call the upload finished endpoint.

This architecture also makes it possible to show a status bar fairly easily.

answered Apr 24 '17 at 23:03

dgatwood

10,129
1
28
49

Unfortunately, unless your data will fit entirely in RAM, I don't think it is possible to avoid creating a truncated copy of the file. If you pre-chunk it, you could create a truncated copy of the current chunk. Either way, your app will have to be woken up and handle the error and restart the transfer. At some point, you're going to start getting penalized by NSURLSession if your app gets woken too frequently, hence chunking is potentially worse than non-chunking. – dgatwood Apr 25 '17 at 16:08
Actually, now that I think about it, you *might* be able to use a stream-upload-based NSURLSessionDownloadTask, but that depends on whether the daemon handles that correctly. Try that approach first, but if it fails, you'll have to create a truncated copy of the file.. – dgatwood Apr 25 '17 at 16:09
I did not get what does mean by the first approach, secondly I tried to make the truncated copy with this approach http://stackoverflow.com/questions/28620522/overwrite-data-using-nsfilehandle but unfortunately, application crashed if file is about 2-3 GB and I have only uploaded few bytes – Fahad Rehman Apr 25 '17 at 17:05
When you create a download task, you can specify an `NSURLRequest` object that contains an `NSStream`. If you use `getBoundStreamsWithBufferSize:inputStream:outputStream:`, your app can be in control over providing the body data. Then, read the data from the file a few megabytes at a time and write it into the output stream, and pass the input stream when creating the `NSURLRequest`. Basically, open the stream for writing before you try to send data, wait until you get a "space available" event, write some data, and check the return value to determine how much data was actually sent. – dgatwood Apr 25 '17 at 17:19
Be sure to close the stream after you write the last chunk, or the upload will never finish. – dgatwood Apr 25 '17 at 17:19

IOS NSURLSession Background Uploading how to upload Large Files in Background?

1 Answers1