How can I perform operations in JavaScript just like we do pipeline of operations in Java streams?

Question

In Java 8 using streams when I chain methods one after the another the execution of operations are performed in pipelined manner.

Example:

List<Integer> nums = Arrays.asList(1,2,3,4,5,6);    
nums.stream().map(x->{
    x = x * x;
    System.out.println("map1="+x);
    return x;
}).map(x->{
    x = x * 3;
    System.out.println("map2="+x);
    return x;
}).forEach(x-> System.out.println("forEach="+x));

Output:-

map1=1
map2=3
forEach=3
map1=4
map2=12
forEach=12
map1=9
map2=27
forEach=27
map1=16
map2=48
forEach=48
map1=25
map2=75
forEach=75
map1=36
map2=108
forEach=108

But when I tried similarly in javascript.The result is different.As in javascript first operation gets completed and then second operation is performed.Example:-

var nums = [1,2,3,4,5,6 ];
nums.map(x => {
  x = (x * x);
  console.log('map1='+x);
  return x;})
  .map(x => {
  x = x * 3;
  console.log('map2='+x);
  return x;})
  .forEach(x=> console.log('forEach='+x));

Output:-

 map1=1
 map1=4
 map1=9
 map1=16
 map1=25
 map1=36
 map2=3
 map2=12
 map2=27
 map2=48
 map2=75
 map2=108
 forEach=3
 forEach=12
 forEach=27
 forEach=48
 forEach=75
 forEach=108

Is there any way in JavaScript to make it performs operations in a pipeline manner, and I get output as in the Java program?

This question ask only how to collect like in JavaScript but not how internal working changes for same type of methods.

Answer 1

Maybe later (or never) you can use the actual experimental pipeline operator |>, which has the following syntax:

expression |> function

Your wanted result could be achieved by taking the functions as separate functions and iterate the stream array for each pipe.

This works only in FF. From version 58: this feature is behind the --enable-pipeline-operator compile flag.

const
    a = x => { x = x * x; console.log("map1=" + x); return x; },
    b = x => { x = x * 3; console.log("map2=" + x); return x; },
    c = x => console.log("forEach=" + x)

var nums = [1, 2, 3, 4, 5, 6];

nums.forEach(v => v |> a |> b |> c);

The same with a pipe as function (function composition enabling piping) with a closure over the wanted functions.

const
    pipe = (...functions) => input => functions.reduce((acc, fn) => fn(acc), input),
    a = x => { x = x * x; console.log("map1=" + x); return x; },
    b = x => { x = x * 3; console.log("map2=" + x); return x; },
    c = x => console.log("forEach=" + x)

var nums = [1, 2, 3, 4, 5, 6],
    pipeline = pipe(a, b, c);

nums.forEach(pipeline);

Answer 2

If you put each function operation into an array, you can iterate over that array with reduce and pass the last calculated value along in the accumulator until the end of the function array is reached:

var nums = [1,2,3,4,5,6 ];
var fns = [
  (x) => {
    x = x * x;
    console.log('map1=' + x);
    return x;
  },
  (x) => {
    x *= 3;
    console.log('map2=' + x);
    return x;
  },
  (x) => {
    console.log(x);
    return x;
  }
];

nums.forEach((num) => {
  fns.reduce((lastResult, fn) => fn(lastResult), num);
  // pass "num" as the initial value for "lastResult",
  // before the first function has been called
});

You can't use nums.map because .map will necessarily iterate through the whole input array before resolving to the mapped output array (after which the mapped output array will then have another .map called on it).

Answer 3

The equivalent to Java's streams are JavaScript's iterators. Iterator objects unfortunately don't have a map method (yet), but you can easily write one yourself (and even install it on the prototype if you want method syntax).

function* map(iterable, f) {
    for (var x of iterable)
        yield f(x);
}

var nums = [1,2,3,4,5,6];
function square(x) {
  x = (x * x);
  console.log('map1='+x);
  return x;
}
function triple(x) {
  x = x * 3;
  console.log('map2='+x);
  return x;
}
for (const x of map(map(nums.values(), square), triple)) {
  console.log('forEach='+x);
}

Also notice that in functional programming, order doesn't matter for pure operations - you shouldn't need to rely on the execution order if you are using map.

Answer 4

Why re-invent from scratch when we have solutions. This functionality is present in lodash/RxJS/stream.js.

Example snippet from lodash:

_.flow(
 _.assign(rows[0]),
 _.omit('blah')
)(foundUser);

// >> {"charData":[],"ok": 1}

However, javascript runs on single thread and so do these libraries. Java streams benefit from multi core systems(in case of parallel). There they can use multiple threads to use all available cores.

Answer 5

I would suggest to use libraries like RxJS. This gives you more control over the kind of processing, should it be sequential, parallel or whatsoever.

Here is an example, which is close to what you expect:

const source = Rx.Observable.from([{name: 'Joe', age: 30}, {name: 'Frank', age: 20},{name: 'Ryan', age: 50}]);
const example = source.map(person => {
  console.log("Mapping1" + person.name)
  return person.name
});
const subscribe = example.subscribe(val => console.log(val));

outputs:

"Mapping1Joe"
"Joe"
"Mapping1Frank"
"Frank"
"Mapping1Ryan"
"Ryan"

Answer 6

The JS code you're using here doesn't have any concept of lazy evaluation - the return value of .map() is already an array value that must have been completely evaluated before its own .map() method can be executed. This isn't an implementation detail of how the interpreter operates, but part of the language definition. (Meanwhile, the Java code's Collection.stream() method returns a Stream object whose contents aren't evaluated yet.)

JavaScript does have the asynchronous/delayed evaluation functionality you want in the form of the Promise object.

The following code would do something like what you want:

var nums = [1,2,3,4,5,6 ];
nums.map(async function(x) {
  x = (x * x);
  console.log('map1='+x);
  return x;
}).map(async function(x) {
  x = await x;
  x = x * 3;
  console.log('map2='+x);
  return x;
}).forEach(async function(x) {
  x = await x;
  console.log('forEach='+x);
});

Now, in practice this will still print the output in the same order as before, because the Promise resolves immediately. However, this time the evaluation of the map function is indeed "lazy" and could in principle occur in any order. To actually test this, we can introduce a delay in the calculation, using an asynchronous sleep() function (from What is the JavaScript version of sleep()?):

function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

var nums = [1,2,3,4,5,6 ];
nums.map(async function(x) {
  x = (x * x);
  await sleep(1); // 1ms delay
  console.log('map1='+x);
  return x;
}).map(async function(x) {
  x = await x;
  x = x * 3;
  console.log('map2='+x);
  return x;
}).forEach(async function(x) {
  x = await x;
  console.log('forEach='+x);
});

Output:

map1=1
map2=3
forEach=3
map1=4
map2=12
forEach=12
map1=9
map2=27
forEach=27
map1=16
map2=48
forEach=48
map1=25
map2=75
forEach=75
map1=36
map2=108
forEach=108

Answer 7

You kinda get the same output, as in the relative values of the sequence of map1, map2 and forEach values are the same.

The difference in order that you're seeing shows the underlying difference between the machine models of the JVM and the JavaScript runtime engine.

JVMs are threaded. JavaScript is not. That means, that your sequence steps in Java can run immediately after the critical number of map operations have occurred.

In JavaScript, the next step is put at the bottom of the execute stack, and every operation at the top must first be executed before getting to the next items.

As you can see, the methods are functionally equivalent, but have different mechanics.