1

Our product uses a Web Browser Control to import local files directly into our web app. When a user clicks "Import" on the software, selected files are saved to a temp file then the Web Browser Control opens to our web app where the files are imported. From there the user can edit properties in the web app.

This works fine with smaller files but once files reach about 10MB and higher an OutOfMemory Exception is thrown.

Here is a simplification of the code we use to retrieve and pass files to the web app:

public string GetFiles() {
   List<DmFile> dmFiles = new List<DmFile>(); // DmFile is a class containing the file bytes and other document information
   foreach (var file in ImportFiles) { // ImportFiles contains the list of class ImportFile
      byte[] fileBytes = File.ReadAllBytes(file.FilePath);
      DmFile dmFile = new DmFile(file.Name, fileBytes);
      dmFiles.Add(dmFile);
   }
   string jsonList = JsonConvert.SerializeObject(dmFiles);
   return jsonList;
}

GetFiles() serializes the file list and passes the JSON to the view model.

Here is a snippet of the view model (javascript) code:

var webControl: any = window.external;
var jsonFilesString = webControl.GetFiles();

We call the Web Browser Control using window.external. Then we call the GetFiles() method directly from the Javascript code to retrieve the JSON so we can use the file bytes.

The OutOfMemory exception occurs after GetFiles() when it attempts to set the JSON to the jsonFilesString.

Any idea on where we can optimize to allow for these larger file sizes or is this a limitation on our method of file transfer (passing JSON through WebBrowserControl)?

Thanks

Marcus Payne
  • 11
  • 1
  • what is your filepath here `File.ReadAllBytes(file.FilePath);` have you tried adding `File.ReadAllBytes(@file.FilePath);` also does the file path end in a backslash `"\"` also where are you setting the `buffer size` – MethodMan Dec 13 '17 at 23:17
  • You are probably hitting the maximum effective string length on your system. [See What is the maximum possible length of a .NET string?](https://stackoverflow.com/q/140468/3744182) for details. Is there any way to avoid that, e.g. by returning a string array instead of a single string? – dbc Dec 13 '17 at 23:37
  • @MethodMan filePath is a full path to the temp location of the file. I am not setting the buffer here. Instead I am reading all of the bytes. – Marcus Payne Dec 14 '17 at 07:32
  • @dbc I haven't tried a string array. I would have to figure out how to break it up (by files, or by each property) – Marcus Payne Dec 14 '17 at 07:34

1 Answers1

1

You will not solve this issue with your current way of transferring file data. You are using an incredible amount of memory converting all those bytes to a string.

Recommendation is to rearrange your application to utilize a pattern that does not require conversion of byte[]'s to string.

1) Separate the process of saving the files and editing the files properties.

2) Provide a form for user to edit file properties. User submits changes to your service, service updates file properties.

3) Only return the binary when the user has a need to save it(download it).

Achievement is a scale-able solution for much larger files at and beyond your 10MB limit.

James
  • 66
  • 2
  • While doing some testing, I separated the document properties from the file when passing in the JSON. Unfortunately it didn't make a difference since the document properties use such little memory in comparison to the file bytes. At this point, a complete rework of this process may not be possible. We would need a better way to transfer files through the WebBrowserControl instead of the bytes through a string. – Marcus Payne Dec 14 '17 at 07:49