FileReader losing data when reading PDF

Question

I have the constraint of sending data to a server in JSON format only, and I need to send a PDF file together with other form data in the JSON. I though I could make a string from it with base64 like in this solution I found on SO:

let data = {foo: 1, bar: 2};
let reader = new FileReader();
reader.readAsDataURL(pdf);
reader.onload = () => {
  data.file = reader.result;
  $.ajax({type: 'POST', dataType: "json", data: JSON.stringify(data), ...});
}

But it happened that reader.result does not contain the whole PDF (whether I save it to file instead of sending, or get it back from the server). In a text editor the content is the same, but a binary editor says that some bytes are missing. My browser can load it as a pdf and the title shows, but the page is blank.

I also tried reader.readAsBinaryString and transform to base64 myself with btoa, but same result.

Edit: CodePen example: https://codepen.io/jdelafon/pen/eRWLdz?editors=1011

Edit: to verify, I did this:

let reader = new FileReader();
reader.readAsBinaryString(pdf);
reader.onload = () => {
    let blob = reader.result;
    let file = new Blob([blob], {type: 'application/pdf'});
    let fileURL = window.URL.createObjectURL(file);
    // make it display in <embed>
};

The body of the pdf is empty. If I replace file by the original pdf, it gets displayed entirely. So FileReader loses data in the process.

Is there a better way? Is it an encoding issue with FileReader?

I also though about using FormData like this:

let data = {foo: 1, bar: 2};
let fd = new FormData();
fd.append('file', pdf);
data.file = btoa(fd);
$.ajax({type: 'POST', dataType: "json", data: JSON.stringify(data), ...});

But now when I fetch the data back from the server, JS has no idea that it represents a FormData object, so I don't know how to get the file back for display in a browser. Is it possible?

Are you sure the data is truncated in the browser, not on the server? — Bergi, Jun 21 '17 at 01:11
@Bergi yes, because I can display it directly in the browser before sending it, and I can compare with the original file, before passing through FileReader. I added the code for that in my question. The original file comes from a . — JulienD, Jun 21 '17 at 01:16
try removing `dataType: "json"` I'm not really sure if it would work. it's just a theory. — perseusl, Jun 21 '17 at 01:17
@perseusl the data is truncated before I send it with Ajax. Maybe I should write a new question without the json context. Plus if it is a string, it should not be different from other strings from the json point of view. — JulienD, Jun 21 '17 at 01:19
@Bergi I do not know for other kinds of files, I will try to test with an image. My pdfs are quite small, below 100KB. — JulienD, Jun 21 '17 at 01:26
Could you somehow share an example file which reproduces this problem ? Do all browsers have the same behavior ? Encoding should not be a problem for readAsDataURL, it would be only for readAsText. Oh and maybe silly question, but do you have an ad-blocker enabled ? — Kaiido, Jun 21 '17 at 02:45
@Kaiido I created a CodePen that reproduces the problem: https://codepen.io/jdelafon/pen/eRWLdz?editors=1011. Ad-block yes, but it works for the original file. Firefox and Chrome have the same problem. — JulienD, Jun 21 '17 at 08:26
But it's not what your question is about... Yes you're doing it wrong, you can't simply pass the result of **deprecated** `readAsBinaryString` into `new Blob([str])`, there will probably be an encoding issue from this binary data. If you really want to do it this way, you need to first convert this binary string to an arrayBuffer, and then create an Blob from the arrayBuffer. Your question states that `readAsDataURL` doesn't work, but I can't repro : https://codepen.io/anon/pen/gRWBxP?editors=1011 — Kaiido, Jun 21 '17 at 08:54
@Kaiido That works, thanks!! You have no idea how long I've been trying. Would you like to copy your CodePen snippet as an answer and I accept it? — JulienD, Jun 21 '17 at 11:55

Watilin · Answer 1 · 2017-06-21T14:09:40.567

1

You weren’t far from succeeding when you tried btoa, except you can’t “btoa-ize” a FormData.

reader.readAsBinaryString(pdf); // don’t use this! See edit below
reader.onload = () => {
  let b64payload = btoa(reader.result);
  $.ajax({
    type: 'POST',
    dataType: "json",
    data: JSON.stringify({ "file": b64payload }),
  });
}

I’m not sure why the readAsDataURL solution failed though.

Edit: although not sure, it’s been suspected that the deprecated readAsBinaryString method might be the source of the problem. The following solution, based on readAsDataURL, is working:

reader.readAsDataURL(pdf);
reader.onload = () => {
  $.ajax({
    type: 'POST',
    dataType: "json",
    data: JSON.stringify({ "dataURL": reader.result }),
  });
};

edited Jun 21 '17 at 14:09

answered Jun 21 '17 at 00:23

Watilin

286
3
9

I tried this too, and yet the PDF gets truncated. I feel like this method sends the "content" of the pdf, not the file itself. To verify, I did in sequence, from your `b64payload`, -> `atob` -> `new Blob` -> `URL.createObjectURL` -> display in ``. The boby of the pdf is empty. The original pdf gets displayed entirely. – JulienD Jun 21 '17 at 00:54
3

Here’s a working example with `readAsDataURL`: https://codepen.io/anon/pen/zzwyeP?editors=1011 Next thing to do is checking if it survives an ajax trip. – Watilin Jun 21 '17 at 10:31
1

Many thanks, that is the example I needed - looks like Kaiido's. I don't see why a b64 string wouldn't survive the trip. I just checked and it did. Awesome. – JulienD Jun 21 '17 at 12:00
So can we conclude the question by saying `readAsBinaryString` was the culprit? Also, we didn’t explore the `readAsArrayBuffer` option. – Watilin Jun 21 '17 at 12:19
Yes, and my misunderstanding of how `readAsDataURL` is actually used. If you post your snippet as answer I can vote it up. – JulienD Jun 21 '17 at 13:25
@MrD It might be that your pdf file is too large. In this case you can’t use a data URL, you have to send the file separately from the json. – Watilin May 06 '19 at 11:53

score 0 · Accepted Answer · answered May 14 '21 at 10:53

Watilin's answer was not added, so here is the working snippet copied from Codepen:

function onUpload(input) {  
  let originalFile = input.files[0];
  let reader = new FileReader();
  reader.readAsDataURL(originalFile);
  reader.onload = () => {
    let json = JSON.stringify({ dataURL: reader.result });

    // View the file
    let fileURL = JSON.parse(json).dataURL;
    $("#display-pdf").empty();
    $("#display-pdf").append(`<object data="${fileURL}"
      type="application/pdf" width="400px" height="200px">
    </object>`);

    // View the original file
    let originalFileURL = URL.createObjectURL(originalFile);
    $("#display-pdf").append(`<object data="${originalFileURL}"
      type="application/pdf" width="400px" height="200px">
    </object>`)
    .onload(() => {
      URL.revokeObjectURL(originalFileURL);
    });
  };
}

HTML:

<input id="file-input" type="file" onchange="onUpload(this)" />
<div id="display-pdf"></div>

FileReader losing data when reading PDF

2 Answers2

Linked