8

I am trying to upload files from Angular 4 app to a JSON API service that accepts base64 strings as file content.

So what I do is - read the file with FileReader.readAsDataURL, then when user confirms the upload I will create a JSON request to the API and send the base64 string of the file I got earlier.

This is where the problem starts - as soon as I do something with the "content" (log it, send it, w/e) the request will be send, but its insanely slow, e.g. 20 seconds for 2MB file.

I have tried:

  • using ArrayBuffer and manually converting it to base64
  • storing the base64 string in HTML and retrieving it later
  • reading the files after user clicks on upload button
  • using the old client from @angular/common
  • using plain XHR request

but everything leads to the same result.

I know where the problem lies. But why does it happen? Is it something browser specific or angular specific? Is there a more preferred approach (keep in mind it has to be base64 string)?


Notes:

  • changing anything in the API is beyond my control
  • API is fine, sending any file trough postman will finish immediately

Code:

This method runs when user adds file to the dropzone:

public onFileChange(files: File[]) : void {
    files.forEach((file: File, index: number) => {
        const reader = new FileReader;

        // UploadedFile is just a simple model that contains filename, size, type and later base64 content
        this.uploadedFiles[index] = new UploadedFile(file);

        //region reader.onprogress
        reader.onprogress = (event: ProgressEvent) => {
            if (event.lengthComputable) {
                this.uploadedFiles[index].updateProgress(
                    Math.round((event.loaded * 100) / event.total)
                );
            }
        };
        //endregion

        //region reader.onloadend
        reader.onloadend = (event: ProgressEvent) => {
            const target: FileReader = <FileReader>event.target;
            const content = target.result.split(',')[1];

            this.uploadedFiles[index].contentLoaded(content);
        };
        //endregion

        reader.readAsDataURL(file);
    });
}

This method runs when users clicks save button

public upload(uploadedFiles: UploadedFile[]) : Observable<null> {
    const body: object = {
        files: uploadedFiles.map((uploadedFile) => {
            return {
                filename: uploadedFile.name,
                // SLOWDOWN HAPPENS HERE
                content: uploadedFile.content
            };
        })
    };

    return this.http.post('file', body)
}
realshadow
  • 2,599
  • 1
  • 20
  • 38
  • You're asking about a problem with your code, but you're not posting any single line of it. – JB Nizet Dec 02 '17 at 10:33
  • @JB Nizet I added the relevant code – realshadow Dec 02 '17 at 11:03
  • What happens if you build the JSON string yourself using simple string concatenation (since you know that base64 doesn't contain any character that must be encoded), instead of letting http do it? – JB Nizet Dec 02 '17 at 11:10
  • Same behaviour. Same thing happens if I sent dummy data to server and try to cosole.log the content, it will take about 20 seconds to show up in console... – realshadow Dec 02 '17 at 11:21
  • 2MB is a huge string to display in the console. – JB Nizet Dec 02 '17 at 11:39
  • I know, just saying it takes about the same time when I do anything (be it logging or request) with the content. – realshadow Dec 02 '17 at 11:44
  • Possible solution https://stackoverflow.com/questions/25810051/filereader-api-on-big-files – Hackerman Dec 04 '17 at 20:23
  • The solution is elegant, however I need "upload" whole file. – realshadow Dec 06 '17 at 19:29
  • angular monkey patches a lot of stuff. Have you tried running this code outside angular's zone ? I mean wrap the upload code with `this._ngZone.runOutsideAngular(() => {` maybe some internal mechanism kicks in and tries to compare it – SebOlens Dec 06 '17 at 22:10
  • Sadly, there is no change. I am sure it is a browser problem with the way the request is handled – realshadow Dec 07 '17 at 07:45

1 Answers1

4

For sending big files to server you should use FormData to be able to send it as multi-part instead of a single big file.

Something like:

// import {Http, RequestOptions} from '@angular/http';
uploadFileToUrl(files, uploadUrl): Promise<any> {
  // Note that setting a content-type header
  // for mutlipart forms breaks some built in
  // request parsers like multer in express.
  const options = new RequestOptions();
  const formData = new FormData();

  // Append files to the virtual form.
  for (const file of files) {
    formData.append(file.name, file)
  }
  // Send it.
  return this.http.post(uploadUrl, formData, options);
    
}

Also don't forget to set the header 'Content-Type': undefined, I've scratched my head over this for hours.

Yaser
  • 5,609
  • 1
  • 15
  • 27
  • This does not solve my problem since I can not change the API. I am aware that this seems as the only solution so I am just looking for an answer to why it takes so much time to handle the request. Thanks anyway – realshadow Dec 06 '17 at 19:28
  • In that case remember the only thing that comes to mind is to use http compression/deflate @realshadow – Yaser Dec 06 '17 at 20:51