JavaScript FileReader Slice Performance

Question

I am trying to access the first few lines of text files using the FileApi in JavaScript.

In order to do so, I slice an arbitrary number of bytes from the beginning of the file and hand the blob over to the FileReader.

For large files this takes very long, even though, my understanding currently is that only the first few bytes of the file need to be accessed.

• Is there some implementation in the background that requires the whole file to be accessed before it can be sliced?
• Does it depend on the browser implementation of the FileApi?

I currently have tested in both Chrome and Edge (chromium).

Analysis in Chrome using the performance dev tools shows a lot of idle time before the reader.onloadend and no increase in ram usage. This might be however, because the FileApi is implemented in the Browser itself and does not reflect in the JavaScript performance statistics.

My implementation of the FileReader looks something like this:

const reader = new FileReader();

reader.onloadend = (evt) => {
  if (evt.target.readyState == FileReader.DONE) {
    console.log(evt.target.result.toString());
  }
};

// Slice first 10240 bytes of the file
var blob = files.item(0).slice(0, 1024 * 10);

// Start reading the sliced blob
reader.readAsBinaryString(blob);

This works fine but as described performs quite underwhelmingly for large files. I tried it for 10kb, 100mb and 6gb. The time until the first 10kb are logged seems to correlate directly to the file size.

Any suggestions on how to improve performance for reading the beginning of a file?

---

Edit: Using Response and DOM streams as suggested by @BenjaminGruenbaum does sadly not improve the read performance.

var dest = newWritableStream({​​​​​​​​
    write(str) {​​​​​​​​
        console.log(str);
    }​​​​​​​​,
}​​​​​​​​);
var blob = files.item(0).slice(0, 1024 * 10);

(blob.stream ? blob.stream() : newResponse(blob).body)
// Decode the binary-encoded response to string
  .pipeThrough(newTextDecoderStream())
  .pipeTo(dest)
  .then(() => {​​​​​​​​
      console.log('done');
  }​​​​​​​​);

Answer 1

Just for kicks, here it is with a worker thread and File System Access API

No idea if either of those things help, I have no 6gb files. This will get the reading off the main thread so that does help performance in some sense.

Object.assign(
    new Worker(
        URL.createObjectURL(
            new Blob(
                [
                    `self.onmessage = async (e) =>` +
                    `    void postMessage(` +
                    `        (new FileReaderSync())` +
                    `            .readAsText(` +
                    `                (await e.data.getFile())` +
                    `                    .slice(0,1024*10)` +
                    `            )` +
                    `    );`
                ],
                { type: 'application/javascript' }
            )
        )
    ),
    { onmessage: (e) => void console.log(e.data) }
).postMessage(
    (await window.showOpenFilePicker(
        { mode: 'read', startIn: 'documents' }
    )).pop()
);

edit:

forgor but you need chromium for this rn sorry (tested on edge) also this wont run in a jsfiddle because web worker blah blah security blah blah. You can copy paste it into the console on google though. for some reason the headers dont prevent thins from running. If this actually does help please actually put the worker in its own file (and reformat my artistic negative space triangle out of existence)

Answer 2

how about this!!

function readFirstBytes(file, n) {
  return new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.onload = () => {
      resolve(reader.result);
    };
    reader.onerror = reject;
    reader.readAsArrayBuffer(file.slice(0, n));
  });
}

readFirstBytes('file', 10).then(buffer => {
  console.log(buffer);
});