How to "pass" extra data between chained maps and filters

Question

This is mostly for academic interest, as I've managed to solve it an entirely different way, but, short story, what I want is, in pseudocode:

Foreach object in array1
    Find matching otherObject in array2 // must exist and there's only 1
    Find matching record in array3      // must exist and there's only 1
    If record.status !== otherObject.status
        push { otherObject.id, record.status } onto newArray

It intuitively seems to me there should be a way to do something with array1.filter(<some function>).map(<some other function>, but I can't get it to work in practice.

Here's a real-world example. This works:

function update(records) {
  const filtered = records.filter((mcr) => {
    const match =  at._people.find((atr) => atr.email.toLowerCase() ===
      mcr.email.toLowerCase());

    return (match.subscriberStatus.toLowerCase() !==
      mc.mailingList.find((listEntry) =>
        listEntry.id === mcr.id).status.toLowerCase()
    );
  });

  const toUpdate = filtered.map((mcr) => {
    const match =  at._people.find((atr) => atr.email.toLowerCase() ===
      mcr.email.toLowerCase());

    return ({ 'id': match.id,
              'fields': {'Mailing List Status': mcr.subscriberStatus }
            }
    );
  });
}

But what bums me out is the duplicated const match =. It seems to me that those could be expensive if at._people is a large array.

I naively tried:

function update(records) {
  let match;

  const toUpdate = records.filter((mcr) => {
    match = at._people.find((atr) => atr.email.toLowerCase() ===
      mcr.email.toLowerCase());

    // return isDifferent?
    return (match.subscriberStatus.toLowerCase() !==
      mc.mailingList.find((listEntry) => listEntry.id === mcr.id).status.toLowerCase());
  }).map((foundMcr) => {
    return ({ 'id': match.id, 'fields': {'Mailing List Status': foundMcr.subscriberStatus } })
  });
}

But (somewhat obviously, in retrospect) this doesn't work, because inside the map, match never changes — it's just always the last thing it was in the filter. Any thoughts on how to pass that match.id found in the filter on an entry-by-entry basis to the chained map? Or, really, any other way to accomplish this same thing?

Answer 1

If you were to only use .map and .filter() then you can avoid extra re-calculations later on in the chain if you do the following (generic steps):

1. .map() each item into a wrapper object that contains:

   • item from the array
   • calculate the extra data you would need in later steps

2. .filter() the wrapper objects based on the calculated data.

3. .map() the leftover results into the shape you wish, drawing on the original item and any of the calculated data.

---

In your case, this can mean that:

1. You do the find logic once.
2. Use the found items to discard some of the results.
3. Use the rest to generate a new array.

Here is the result with the callbacks extracted to make the map/filter/map logic clearer:

//takes a record and enriches it with `match` and `mailingStatus`
const wrapWithLookups = mcr => {
    const match =  at._people.find((atr) => atr.email.toLowerCase() ===
      mcr.email.toLowerCase());
    const mailingListStatus = mc.mailingList.find((listEntry) => listEntry.id === mcr.id).status;

    return { match, mailingListStatus , mcr };
};

//filters based on match and mailingListStatus calculated fields
const isCorrectSubscriberStatus = ({match, mailingListStatus}) => 
    match.subscriberStatus.toLowerCase() !== mailingListStatus .toLowerCase();

//converts to a new item based on mcr and match
const toUpdatedRecord = ({match, mcr}) => ({
    'id': match.id,
    'fields': {'Mailing List Status': mcr.subscriberStatus }
});

function update(records) {
    return records
        .map(wrapWithLookups)
        .filter(isCorrectSubscriberStatus)
        .map(toUpdatedRecord);
}

---

This saves the re-calculation of match and/or mailingStatus if they are needed later. However, it does introduce an entire new loop through the array just to collect them. This could be a performance concern, however, it is very easily remedied if you use lazy evaluated chain like what is provided by Lodash. The code adjustment to use that would be:

function update(records) {
    return _(records) // wrap in a lazy chain evaluator by Lodash ->-+
        .map(wrap)                         // same as before         |
        .filter(isCorrectSubscriberStatus) // same as before         |
        .map(toUpdatedRecord)              // same as before         |
        .value(); // extract the value <-----------------------------+
}

Other libraries would likely have a very similar approach. In any case, lazy evaluation does not run once through the array for .map(), then another time for .filter(), then a third time for the second .map() but instead only iterates once and runs the operations as appropriate.

---

Lazy evaluation can be expressed through a transducer which is built on top of reduce(). For an example of how transducers work see:

How to chain map and filter functions in the correct order
Transducer flatten and uniq

Thus it is possible to avoid all the .map() and .filter() calls by simply doing one combined function and directly use .reduce(). However, I personally find that harder to reason about and more difficulty to maintain, than expressing the logic through a .map().filter().map() chain and then using lazy evaluation, if performance is needed.

---

Worth noting that the map() -> filter() -> map() logic does not need to used via a lazy chain. You can use a library like the FP distribution of Lodash or the vanilla Ramda that give you generic map() and filter() functions that can be applied to any list and combined with each other to avoid multiple repetitions again. Using Lodash FP this would be:

import map from 'lodash/fp/map';
import filter from 'lodash/fp/filter';
import flow from 'lodash/fp/flow';

function update(records) {
    const process = flow(
        map(wrapWithLookups),
        filter(isCorrectSubscriberStatus),
        map(toUpdatedRecord),
    );
    
    return process(records);
}

For Ramda the implementation would be the same - map() and filter() act the same across the two libraries, the only difference is that the composition function (flow() in Lodash) is called pipe in Ramda. It acts identically to flow():

pipe(
    map(wrapWithLookups),
    filter(isCorrectSubscriberStatus),
    map(toUpdatedRecord),
)

For a deeper look at why you might want to avoid chaining and the alternative here, see the article on Medium.com: Why using _.chain is a mistake.

Answer 2

Here's how I would do this, if this can help:

let newArray = array1.filter((item) => {
   let matching1 = array2.filter(matchingFunction)  
   let matching2 = array3.filter(matchingFunction) 
   return matching1?.status == matching2?.status; 
})