How can I efficiently write numeric data to a file?

Question

Say I have an array containing a million random numbers:

[ 0.17309080497872764, 0.7861753816498267, ...]

I need to save them to disk, to be read back later. I could store them in a text format like JSON or csv, but that will waste space. I'd prefer a binary format where each number takes up only 8 bytes on disk.

How can I do this using node?

UPDATE

I did not find an answer to this specific question, with a full example, in the supposedly duplicate question. I was able to solve it myself, but in a verbose way that could surely be improved:

// const a = map(Math.random, Array(10));
const a = [ 
  0.9651891365487693,
  0.7385397746441058,
  0.5330173086062189,
  0.08100066198727673,
  0.11758119861500771,
  0.26647845473863674,
  0.0637438360410223,
  0.7070151519015955,
  0.8671093412761386,
  0.20282735866103718
];

// write the array to file as raw bytes (80B total)
var wstream = fs.createWriteStream('test.txt');
a.forEach(num => {
  const b = new Buffer(8);
  b.writeDoubleLE(num);
  wstream.write(b);
})
wstream.end(() => {
  // read it back
  const buff = fs.readFileSync('test.txt');
  const aa = a.map((_, i) => buff.readDoubleLE(8*i));
  console.log(aa);
});

Possible duplicate of [Read/Write bytes of float in JS](http://stackoverflow.com/questions/4414077/read-write-bytes-of-float-in-js) — Pablo Lozano, Jan 31 '17 at 08:24

score 0 · Answer 1 · edited May 23 '17 at 12:09

0

I think this was answered in Read/Write bytes of float in JS

The ArrayBuffer solution is probably what you are looking for.

edited May 23 '17 at 12:09

Community

1
1

answered Jan 31 '17 at 07:56

ephraim

21
2

How can I efficiently write numeric data to a file?

UPDATE

1 Answers1