Javascript streams library - implementation

Question

java streams (or any other functional library for other languages) are very nice.

For example, you can have (js sudo code).

Stream.of([1, 2, 3]).filter(x => x > 2).map(x => x * 5).result(); // [15]

Ignore the syntax or the specific implementation it's just an example.

Now my problems are when the flow is a little bit complicated.

For example, if I need different data on each step like that:

Stream.of([1,2, 3])
  .map(x => x * 3)
  .zip([4, 5, 6])
  .map(..//here i need the initial array)
  .map(..//here i need the zipped array)
  .total(..//

As you see in some methods i need the last calculated value, in some i need the initial value.

Also, there are situations where I need the intermediate values but after they are calculated.

map(x => x * 1).map(x => x * 2).map(x => x * 4).map(..//i need the result from 2nd map (x*2)

This is a silly example but illustrates the problem.

Is there a good solution to this problem.

I thought I can save all data in the object but this leads to more verbose code because on each step I have to set and get the properties from the object.

Another example: Sum the numbers: [1, 2, 3, 4] -> 10 Filter numbers above 2: [1, 2, 3, 4] -> [3, 4] Multiple each number with the sum: [30, 40]

  Stream.of([1,2,3, 4])
    .sum()
    .filter(// here will be the sum, but i want the initial array and later the sum)
    .map(// here i want the filtered array and the calculated sum)

Thanks

Which library/language provides `zip` here? Its not a default from JDK for sure. Also, if you need the initial array, maybe choose to implement the former `map` later. — Naman, Mar 07 '19 at 09:30
Stream.js but the issue is in all. If i want for example to map the initial array, then zip the result in another array and map the result with the initial array or intermediate result. This can be array, the sum of the elements for example. I far as I understand on each step I lose the previous value so If want to get it later can I do that. Thanks — , Mar 07 '19 at 09:35
It's about language-agnostic functional programming theory though, isn't it? — DodgyCodeException, Mar 07 '19 at 09:46
Possible duplicate of [How can I perform operations in JavaScript just like we do pipeline of operations in Java streams?](https://stackoverflow.com/questions/54568053/how-can-i-perform-operations-in-javascript-just-like-we-do-pipeline-of-operation) — Jaspreet Jolly, Mar 07 '19 at 10:03

score 2 · Answer 1 · answered Mar 08 '19 at 08:34

2

If you need an intermediate result, then save the computation:

const initial = Stream.of([1,2,3,4]);
const total   = initial.sum();
const result  = initial.filter(x => x > 2).map(x => x * total);

The above example is the most logical way to write such code. I don't see why you'd want to write code like:

Stream.of([1,2,3, 4])
    .sum()
    .filter(/* here will be the sum, but i want the initial array and later the sum */)
    .map(/* here i want the filtered array and the calculated sum */)

Your example is confusing and misleading. Code written in a functional style doesn't need to be chained.

answered Mar 08 '19 at 08:34

Aadit M Shah

72,912
30
168
299

1

Just a quick question, since I am unfamiliar with the JavaScript Stream API. In Java this would not work because the second line `total = initial.sum()` would consume the stream and the third line would lead to an Exception. Is this code really correct in JavaScript? Does JavaScript support multiple usages of Streams? – Stefan Winkler Mar 08 '19 at 11:36
It depends upon the implementation of the stream. If the stream is implemented in a purely functional way, as most stream libraries ought to be, then it will work. I'll update my answer to show how to do this. – Aadit M Shah Mar 08 '19 at 11:43
It doesn't depend on implementation. It's in the [documentation](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/stream/package-summary.html): "**The elements of a stream are only visited once during the life of a stream. Like an Iterator, a new stream must be generated to revisit the same elements of the source.**" (I assume you're using Java streams in your answer, but if JavaScript has a 3rd-party library that works like that, it would be good to mention it as it doesn't seem to be in standard JS.) – DodgyCodeException Mar 08 '19 at 11:47
@DodgyCodeException Those are Java streams. We're talking about JavaScript, which doesn't have a built-in Stream interface. Hence, we can implement the Stream interface any way we want including a purely functional way. – Aadit M Shah Mar 08 '19 at 11:54
@DodgyCodeException I think the confusion is because the OP mentions both Java streams and JavaScript streams in his question. However, he clearly wants to use streams in JavaScript. – Aadit M Shah Mar 08 '19 at 11:56
It seems more elegant the second style. No variables are created. But it's not a big deal i guess – Mar 08 '19 at 12:36
@user2693928 Yes, but it doesn't type check either. – Aadit M Shah Mar 08 '19 at 14:17

score 1 · Answer 2 · answered Mar 07 '19 at 09:55

I don't know for all the languages--I am mostly a Java developer--but in my understanding, no, there is no silver bullet like the one you are searching.

The metaphor of a Stream as it is used in these types of library is always somthing like this

[input stream] --{read elements one by one}--> [calculate] --> [output stream]

Think of [input stream] not as a view of a collection (although in 99% it is used like this). Consider it more generally as, e.g., a network socket from which you read the elements one by one. And after you've read an element, it's gone from the network socket and you cannot rewind.

So, the basic principle is that

each element is read exactly once,
all [calculate] processes can potentially run in parallel, and
the [calculate] processes do not have side effects

This principle allows for the library to optimize and parallelize your calculations internally, which is the main goal for Stream APIs.

So, to answer your question: If you need a calculation which requires accessing the element of a stream more than once, you need to save the original stream or an intermediate result in a collection.

Thanks, I understand. So the procedural code for the example above is like this: let arr = [1, 2, 3, 4] let sum = arr.reduce((acc,x) => x + acc, 0) let filtered = arr.filter(x => x > 2) let result = filtered.map(x => x * sum) If I want to achieve the same result with Stream it will be something like this? Stream.of({nums: [1, 2, 3, 4]}) .map(o => {...o, sum: o.nums.reduce((acc, x) => acc + x, 0)}) .map(o => {...o, filtered: o.nums.filter(x => x > 2)}) .map(o => o.filtered.map(x => x * o.sum)) — , Mar 07 '19 at 10:13
You could probably write it like this, but it would be an awful misuse of the concept IMHO. Just because you have a concept (Streams in this case) in hand, it does not mean you need to write everything with it. I think the proficiency in programming is using the right tools, libraries, APIs, and yes, also programming styles or paradigms for the task at hand. See the other answer of Aadit for a good compromise of using streams in procedural context. — Stefan Winkler, Mar 08 '19 at 11:32

Javascript streams library - implementation

2 Answers2