How do I sort stream via. passthrough and pipe in node js? I am trying to sort from a .fs.readstream and pass it through a passthrough steam that sorts the data from the readstream and pipe it to a writestream. I am unable to find any examples of this online.
Asked
Active
Viewed 93 times
0
-
What do you mean with sorting? To wait for all elements, then sort them and pass them on as a single element? If that's the case, would it work for you to collect all items and then simply write it afterwards? (for collecting all items to an array, see e.g. https://stackoverflow.com/a/49428486/6548154) – A_A Dec 12 '22 at 19:15
-
I need to do a binary search for a pivot array from a 2GB+ file. The only way I will ever get the search done in time is via. a binary search. However that requires that the readstream from the file is sorted. – ryokan Dec 12 '22 at 19:45
-
I'm not sure that I understand what you mean with a pivot array. However, does that mean you want to load the 2GB+ file into memory, *then* do sorting there and then binary search? If yes, does the previous solution work for you? I wouldn't see the point to handle everything in a stream – A_A Dec 12 '22 at 19:52
-
Can you provide an example input stream, how the sorting takes place (chunks of the stream? the whole data at once? a specific sorting algorithm?) and example output? I think that would help people to understand the problem and write an answer – A_A Dec 12 '22 at 19:54