I was downloading some files and writing it to a file which is opened as writeStream but when I tried to delete it , it got deleted but fs was still doing its job but writing nowhere... Where that buffer is going?? as I deleted that file in which it was writing
Asked
Active
Viewed 522 times
2 Answers
3
If you are on a UN*X-y OS, a deleted file continues to exist until all file descriptors open to it are closed. It exists, but it has no name.
"What happens to an open file handle on Linux if the pointed file gets moved or deleted" gives details on Linux.
ariels
- 545
- 4
- 17
-
Thanks for the anser @ariels but i didn't understand one thing.. as i unlinked the file , the fs is still writing chunks to heap and heap is cleared when the file handle is closed? right!... if that's true then is there any way to tell fs not to write on the buffer as file got delete?? – AxDu Apr 07 '20 at 13:32
-
Not on the OS side of things: "[the file] exists, but it has no name". Your process might write the file, then seek across it and read it. You can still do anything you would like to do programmatically, of course – ariels Jul 23 '20 at 06:19
0
I think the general answer is JS just keeps the writing data chunks in the heap and release them at the proper moment under its garbage collection mechanism.
The writing data chunk could be one of types <string> | <Buffer> | <Uint8Array> ..., maybe you want to have a look on API
Buffer.allocUnsafeSlow(size), it mentions:
When using Buffer.allocUnsafe() to allocate new Buffer instances, allocations under 4KB are sliced from a single pre-allocated Buffer. This allows applications to avoid the garbage collection overhead of creating many individually allocated Buffer instances.
According to this paragraph, you can infer allocating memory for Buffer (Which is one of types I mentioned above) by another method Buffer.alloc() is under the garbage collection of Node engine.
The memory handling of JS Object is in the heap. The createStream must open and load the data of the dest file to the heap/memory first, even method like writeStream.write(), it doesn't mean it's directly doing I/O to the file/disk, it must handle the data chunk by chunk in the memory first, then use the least number of times to write it into the disk, so when the dest file is gone, probably the practice of this method just keeps these data chunks in the heap then see when is the good time to release them.
Compare to the situation the dest file still exists, I think there is no big difference, it would end up cleaning the memory for those data.
If you want to know more about garbage collection in Node engine: a-tour-of-v8-garbage-collection
Carr
- 2,691
- 1
- 19
- 27
-
-
@Bergi, actually I am not sure if my **Buffer** relates to its ` Where that buffer is going??` lol, this question could be very general or specific, tbh, I am not sure, just depends on my speculation, I think maybe my answer can be helpful to sort out something? :) – Carr Apr 06 '20 at 12:12
-
I'm pretty sure that OP was talking about the os file buffer that the stream writes to, not the nodejs `Buffer`. – Bergi Apr 06 '20 at 12:36
-
@Bergi, actually I am a bit confused about what _os file buffer_ means and willing to learn now. – Carr Apr 06 '20 at 12:45
-
so according to your answer @Carr, if i somehow deleted that on going process of fs which is writing chunks from heap to that file... how can I clear that buffer?? – AxDu Apr 07 '20 at 13:27
-
1@AxDu, here is a tool that maybe you would be interested: https://github.com/felixge/node-memory-leak-tutorial – Carr Apr 07 '20 at 13:59
-
1@AxDu, I saw the question you commented under the other answer, I think fs does provide API to check if the file still exists, you would be interested in this [thread](https://stackoverflow.com/questions/4482686/check-synchronously-if-file-directory-exists-in-node-js), so I believe you can implement the logic to stop writing to inexistent file by yourself. – Carr Apr 08 '20 at 09:31