0

I'm running a nodeJS program, which initially reads a db, and then writes to it, through several iterations of a for loop

(async ()=>{
const db = JSON.parse(await fs.readFile(DB_URL));
// ... more code
for(let i = 0; i < n /*n in the range of 10^3*/; i++){
  // ... manipulate the db
  // ... more code
  const dir = './FolderName';
  await fs.mkdir(dir, { recursive: true });
  await Promise.all(
    arr.map(async (arrItem, index) =>
      fs.writeFile(`${dir}/${index + 1}.png`, arrItem)
    )
  );
  await fs.writeFile(
    `${dir}/fileName.json`,
    JSON.stringify(db)
  );
}
})()

However, at random times, I'm getting this error.

(node:36680) Warning: Closing file descriptor 39 on garbage collection
(Use `node --trace-warnings ...` to show where the warning was created)
internal/process/warning:50
(node:36680) [DEP0137] DeprecationWarning: Closing a FileHandle object on garbage collection is deprecated. Please close FileHandle objects explicitly using FileHandle.prototype.close(). In the future, an error will be thrown if a file descriptor is closed during garbage collection.

I read from this answer, that fileHandle.close() is necessary only when I am opening a file using fileHandle.open(). Then what is going wrong? Why am I getting this error?

sayandcode
  • 1,775
  • 1
  • 11
  • 24
  • How large is `arr`? Both in terms of number of items and approximate memory size. Possibly the GC is running while some files have already been written but not yet cleaned up due to excessive memory/filedescriptor use. – robertklep Jun 24 '22 at 14:44
  • @robertklep `arr` is 4 images of 300kb each approx – sayandcode Jun 25 '22 at 04:12
  • Hmm okay, that should rule out excessive memory use. – robertklep Jun 25 '22 at 06:05
  • @robertklep Would there be any advantage if I used fileStream? – sayandcode Jun 25 '22 at 06:46
  • I don't think it would help. To be honest, since it's just a warning I would probably ignore it myself :D – robertklep Jun 25 '22 at 06:55
  • @robertklep No it isn't just a warning. The program just freezes when this warning pops. Rerunning the program, closing VScode, and even restarting my computer doesn't seem to help – sayandcode Jun 25 '22 at 07:01
  • Can you log how many fs operations are started in parallel? I guess log the number of elements in `arr` before the `Promise.all()`, and also a "done" message afterwards. If your program freezes it's pointing to excessive use of either RAM or I/O (/file descriptors). – robertklep Jun 25 '22 at 07:24
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/245907/discussion-between-sayandcode-and-robertklep). – sayandcode Jun 25 '22 at 09:52

0 Answers0