Can I close a node readStream before end?

Question

I am dropping a fs.createReadStream onto a large file.

Can I stop the stream part way through?

I don't see one in the doco, and tried calling stream.end() but it still processes the whole file, and stream.close() is undefined.

Any help would be great, thanks.

As @vkurchatkin says, `close()` should do what you want, which is close the underlying file so no more data is read. What are you seeing that contradicts that? — loganfsmyth, Nov 26 '13 at 04:20
I have to get back to this once I get more time to test things out. — blu, Dec 03 '13 at 10:00
possible duplicate of [How to close a readable stream?](http://stackoverflow.com/questions/19277094/how-to-close-a-readable-stream) — Yves M., Apr 22 '15 at 17:54

score 4 · Answer 1 · edited May 23 '17 at 11:51

4

stream.destroy() might be what you want. More details here.

edited May 23 '17 at 11:51

Community

1
1

answered Apr 27 '16 at 03:00

Chad Johnson

21,215
34
109
207

score 1 · Answer 2 · edited May 23 '17 at 11:51

According to How to close a readable stream (before end)?, it is possible, you can call close() on a readStream and it will close before end. I write a simple script to test it:

var fs = require('fs');

var l = 0

var rs = fs.createReadStream("some large file")
    .on("data", function(data){
        console.log("got data");
        l += data.length;
        if (l > 655360) {
            rs.close();
            console.log("close");
        }
    })
    .on("end", function(){
        console.log("shouldn't be logged");
    });

output:

got data
got data
got data
got data
got data
got data
got data
got data
got data
got data
got data
close
got data
close

Though close is printed twice, it does end the readStream.

Shawn · Answer 3 · 2013-11-25T21:06:42.270

0

Edit:

It is not possible at this time by design as can be seen by the code comments: https://github.com/joyent/node/blob/master/lib/_stream_readable.js#L884

A possible solution according to the docs is to use offsets: http://nodejs.org/api/fs.html#fs_fs_createreadstream_path_options

An example to read the last 10 bytes of a file which is 100 bytes long: fs.createReadStream('sample.txt', {start: 90, end: 99});

edited Nov 25 '13 at 21:06

answered Nov 25 '13 at 20:10

Shawn

3,583
8
46
63

I am not looking to read predetermined bytes lengths or offsets; I am doing other logic based on content. Thanks for the answer though. – blu Nov 25 '13 at 20:11
Can you just iterate over the stream until you find what you are looking for? i.e read a certain amount of bytes. If its not there, change the offset and try again? – Shawn Nov 25 '13 at 20:27
I would prefer to do as the question asks: close an open stream. – blu Nov 25 '13 at 20:30
You won't be able to do that with a stream. A stream is an all or nothing IO process. It knows nothing pre-determinant about the data you are looking at. In the case of the offset for example it is going to scan the file to determine length and then retrieve the byte(s) you want. Thus stopping a stream "in the middle" leaves you with corrupt data that is useless. – Shawn Nov 25 '13 at 20:32
It isn't useless if I already pulled out what I need. What about the scenario where I am in non-flowing mode and instead of calling read when I am ready for data, I say we are never going to be ready for that data, just stop streaming data. Can you find in docs where a stream is designed to be an, "all or nothing IO process"? – blu Nov 25 '13 at 20:38
It does look it is designed to always read the whole content: https://github.com/joyent/node/blob/master/lib/_stream_readable.js#L884 – blu Nov 25 '13 at 20:43
I had not found it in the documents; A stream is a construct across all coding languages. I am glad you found that though. – Shawn Nov 25 '13 at 20:48
Other languages allow streams to be closed prior to completion though: http://msdn.microsoft.com/en-us/library/aa328800(v=vs.71).aspx, and Java has one too – blu Nov 25 '13 at 20:53
If you want to modify your answer to say that it isn't possible at this time by design I will accept this answer. If you want to toss in the alternative approach of using the byte offsets it will make a complete answer. – blu Nov 25 '13 at 20:59
Java uses a form of byte blocking and buffering to allow this. Looks to be the same for .NET. You can break off after each block. I also updated the answer with your code reference. – Shawn Nov 25 '13 at 21:08
@Shawn Line 884 is saying that the `end` event will never fire if there is still data to be read, which is true. You can still use `.close` to close the stream early, it will just never fire `end`. – loganfsmyth Nov 26 '13 at 04:24

Can I close a node readStream before end?

3 Answers3