creating top 5 aggregation with ramdajs

Question

I would like to transform this input

[
        { country: 'France', value: 100 },
        { country: 'France', value: 100 },
        { country: 'Romania', value: 500 },
        { country: 'England', value: 400 },
        { country: 'England', value: 400 },
        { country: 'Spain', value: 130 },
        { country: 'Albania', value: 4 },
        { country: 'Hungary', value: 3 }
]

into the output

[
      { country: 'England', value: 800 },
      { country: 'Romania', value: 500 },
      { country: 'France', value: 200 },
      { country: 'Spain', value: 130 },
      { country: 'Other', value: 8 }
]

Which is basically doing a sum of values for the top 4 + others countries.

I am using javascript with ramdajs, and I only managed to do it in a somehow cumbersome way so far.

I am looking for an elegant solution: any Functional Programmer out there able to provide their solution? Or any idea of ramda methods that would help?

and does the answer required to use that library's function only? — apple apple, May 08 '19 at 09:46
I just edited the question to add a link to a runable version of my solution. Regarding the library, I'm looking for the most functional approach to resolve this issue, and I can't introduce any other functional library such as Iodash just for this algorithm but native javascript is ok. — Pierre-Jean, May 08 '19 at 09:49
possible duplicate https://stackoverflow.com/questions/42590750/how-to-get-distinct-values-and-sum-the-total-on-json-using-js — Carlos Alves Jorge, May 08 '19 at 10:05
@CarlosAlvesJorge: I would suggest that this is not a duplicate of that. Yes, you probably want the grouping behavior, but there is much more involved here, including an additional sort, splitting into top-4 and others groups, then combining the others. This is a significantly more complex requirement. — Scott Sauyet, May 08 '19 at 13:08
I love the variety of solutions this question has generated! — Scott Sauyet, May 08 '19 at 17:18
@ScottSauyet To be honest me too...but I don't know how which one to accept now... I hope votes will help me to decide on one :-) — Pierre-Jean, May 08 '19 at 22:01
Please don't use the votes for that. Choose the one that best suits you, in whatever manner. Mine is one of the most concise. appleapple's has a longer pipeline but the simplest steps. customcommander has by far the most detailed exposition. Choose the best answer for you. — Scott Sauyet, May 09 '19 at 00:17

customcommander · Answer 1 · 2019-05-08T16:18:21.203

(Each step gets the output of the previous step. Everything will be put together in the end.)

Step 1: Get a map of sums

You can transform this:

[
  { country: 'France', value: 100 },
  { country: 'France', value: 100 },
  { country: 'Romania', value: 500 },
  { country: 'England', value: 400 },
  { country: 'England', value: 400 },
  { country: 'Spain', value: 130 },
  { country: 'Albania', value: 4 },
  { country: 'Hungary', value: 3 }
]

into this:

{
  Albania: 4,
  England: 800,
  France: 200,
  Hungary: 3,
  Romania: 500,
  Spain: 130
}

With this:

const reducer = reduceBy((sum, {value}) => sum + value, 0);
const reduceCountries = reducer(prop('country'));

Step 2: Convert that back into a sorted array

[
  { country: "Hungary", value: 3 },
  { country: "Albania", value: 4 },
  { country: "Spain", value: 130 },
  { country: "France", value: 200 },
  { country: "Romania", value: 500 },
  { country: "England", value: 800 }
]

You can do this with:

const countryFromPair = ([country, value]) => ({country, value});
pipe(toPairs, map(countryFromPair), sortBy(prop('value')));

Step 3: Create two sub groups, the non-top-4 countries and the top-4 countries

[
  [
    { country: "Hungary", value: 3},
    { country: "Albania", value: 4}
  ],
  [
    { country: "Spain", value: 130 },
    { country: "France", value: 200 },
    { country: "Romania", value: 500 },
    { country: "England", value: 800 }
  ]
]

Which you can do with this:

splitAt(-4)

Step 4: Merge the first sub group

[
  [
    { country: "Others", value: 7 }
  ],
  [
    { country: "Spain", value: 130 },
    { country: "France", value: 200 },
    { country: "Romania", value: 500 },
    { country: "England", value: 800 }
  ]
]

With this:

over(lensIndex(0), compose(map(countryFromPair), toPairs, reduceOthers));

Step 5: Flatten the entire array

[
  { country: "Others", value: 7 },
  { country: "Spain", value: 130 },
  { country: "France", value: 200 },
  { country: "Romania", value: 500 },
  { country: "England", value: 800 }
]

With

flatten

Complete working example

const data = [
  { country: 'France', value: 100 },
  { country: 'France', value: 100 },
  { country: 'Romania', value: 500 },
  { country: 'England', value: 400 },
  { country: 'England', value: 400 },
  { country: 'Spain', value: 130 },
  { country: 'Albania', value: 4 },
  { country: 'Hungary', value: 3 }
];

const reducer = reduceBy((sum, {value}) => sum + value, 0);
const reduceOthers = reducer(always('Others'));
const reduceCountries = reducer(prop('country'));
const countryFromPair = ([country, value]) => ({country, value});

const top5 = pipe(
  reduceCountries,
  toPairs,
  map(countryFromPair),
  sortBy(prop('value')),
  splitAt(-4),
  over(lensIndex(0), compose(map(countryFromPair), toPairs, reduceOthers)),
  flatten
);

top5(data)

`reduceBy`: I knew I was missing something! Very nice technique, and an excellent answer. — Scott Sauyet, May 08 '19 at 13:30
Thanks @ScottSauyet. Looking at your answer using `zipObj` to reconstruct an object from a pair feels more natural than what I'm doing. Although I'm not fully grokking `lift` yet, it also feels more idiomatic in this case. I'll consider this too. — customcommander, May 08 '19 at 13:41
This [discussion of `lift`](https://stackoverflow.com/q/36558598/1243641) may help. It converts a function operating of values to one operating on *containers* of those values. While this could be something like `Maybe`, it can also be a function returning that value. That's what I use here. With functions, `lift(f)(g, h)` is something like `(...args) => f(g(...args), h(...args))`. This is much like `converge`, but with a more standard (and slightly less flexible) behavior. — Scott Sauyet, May 08 '19 at 13:55

Scott Sauyet · Answer 2 · 2019-05-08T13:29:00.027

Here's an approach:

const combineAllBut = (n) => pipe(drop(n), pluck(1), sum, of, prepend('Others'), of)

const transform = pipe(
  groupBy(prop('country')),
  map(pluck('value')),
  map(sum),
  toPairs,
  sort(descend(nth(1))),
  lift(concat)(take(4), combineAllBut(4)),
  map(zipObj(['country', 'value']))
)

const countries = [{ country: 'France', value: 100 }, { country: 'France', value: 100 }, { country: 'Romania', value: 500 }, { country: 'England', value: 400 }, { country: 'England', value: 400 }, { country: 'Spain', value: 130 }, { country: 'Albania', value: 4 }, { country: 'Hungary', value: 3 }]

console.log(transform(countries))

<script src="https://bundle.run/ramda@0.26.1"></script>
<script>
const {pipe, groupBy, prop, map, pluck, sum, of, prepend, toPairs, sort, descend, nth, lift, concat, take, drop, zipObj} = ramda
</script>

Except for the one complex line (lift(concat)(take(4), combineAllBut(4))) and the associated helper function (combineAllBut), this is a set of simple transformations. That helper function is probably not useful outside this function, so it would be perfectly acceptable to inline it as lift(concat)(take(4), pipe(drop(4), pluck(1), sum, of, prepend('Others'), of)), but I find the resulting function a little too difficult to read.

Note that that function will return something like [['Other', 7]], which is a format meaningless outside the fact that we're going to then concat it with an array of the top four. So there's at least some argument for removing the final of and replacing concat with flip(append). I didn't do so since that helper function means nothing except in context of this pipeline. But I would understand if someone would choose otherwise.

I like the rest of this function, and it seems to be a good fit for the Ramda pipeline style. But that helper function spoils it to some degree. I would love to hear suggestions for simplifying it.

Update

Then answer from customcommander demonstrated a simplification I could take, by using reduceBy instead of of the groupBy -> map(pluck) -> map(sum) dance in the above approach. That makes for a definite improvement.

const combineAllBut = (n) => pipe(drop(n), pluck(1), sum, of, prepend('Others'), of)

const transform = pipe(
  reduceBy((a, {value}) => a + value, 0, prop('country')),
  toPairs,
  sort(descend(nth(1))),
  lift(concat)(take(4), combineAllBut(4)),
  map(zipObj(['country', 'value']))
)

const countries = [{ country: 'France', value: 100 }, { country: 'France', value: 100 }, { country: 'Romania', value: 500 }, { country: 'England', value: 400 }, { country: 'England', value: 400 }, { country: 'Spain', value: 130 }, { country: 'Albania', value: 4 }, { country: 'Hungary', value: 3 }]

console.log(transform(countries))

<script src="https://bundle.run/ramda@0.26.1"></script>
<script>
const {pipe, reduceBy, prop, map, pluck, sum, of, prepend, toPairs, sort, descend, nth, lift, concat, take, drop, zipObj} = ramda
</script>

apple apple · Accepted Answer · 2019-05-08T13:55:57.500

I give it a try and try to use it's function for most things. and keep it single pipe

const f = pipe(
  groupBy(prop('country')),
  map(map(prop('value'))),
  map(sum),
  toPairs(),
  sortBy(prop(1)),
  reverse(),
  addIndex(map)((val, idx) => idx<4?val:['Others',val[1]]),
  groupBy(prop(0)),
  map(map(prop(1))),
  map(sum),
  toPairs(),
  map(([a,b])=>({'country':a,'value':b}))
)

Ramda REPL

However, I don't think it's any way readable.

score 1 · Answer 4 · answered May 08 '19 at 10:33

I think you can slightly simplify groupOthersKeeping by splitting the array before reducing it, in terms of ramda, that may look like as follows:

const groupOthersKeeping = contriesToKeep => arr => [
    ...slice(0, contriesToKeep, arr),
    reduce(
      (acc, i) => ({ ...acc, value: acc.value + i.value }),
      { country: 'Others', value: 0 },
      slice(contriesToKeep, Infinity, arr)
    )
 ]

score 1 · Answer 5 · answered May 08 '19 at 10:36

Using more ramda function but not sure that is better:

let country = pipe(
  groupBy(prop('country')),
  map(pluck('value')),
  map(sum)
)([
  { country: 'France', value: 100 },
  { country: 'France', value: 100 },
  { country: 'Romania', value: 500 },
  { country: 'England', value: 400 },
  { country: 'England', value: 400 },
  { country: 'Spain', value: 130 },
  { country: 'Albania', value: 4 },
  { country: 'Hungary', value: 3 }
]);

let splitCountry = pipe(
  map((k) => ({country: k, value: country[k]})),
  sortBy(prop('value')),
  reverse,
  splitAt(4)
)(keys(country));

splitCountry[0].push({country: 'Others', value: sum(map(prop('value'))(splitCountry[1]))});
splitCountry[0]

Krantisinh · Answer 6 · 2019-05-08T12:00:03.403

Here are my two cents.

const a = [
    { country: 'France', value: 100 },
    { country: 'France', value: 100 },
    { country: 'Romania', value: 500 },
    { country: 'England', value: 400 },
    { country: 'England', value: 400 },
    { country: 'Spain', value: 130 },
    { country: 'Albania', value: 4 },
    { country: 'Hungary', value: 3 }
];

const diff = (a, b) => b.value - a.value;
const addValues = (acc, {value}) => R.add(acc,value);
const count = R.reduce(addValues, 0);
const toCountry = ({country}) => country;
const toCountryObj = (x) => ({'country': x[0], 'value': x[1] });
const reduceC = R.reduceBy(addValues, [], toCountry);

const [countries, others] = R.compose(
    R.splitAt(4), 
    R.sort(diff), 
    R.chain(toCountryObj), 
    R.toPairs, 
    reduceC)(a);

const othersArray = [{ 'country': 'Others', 'value': count(others) }];

R.concat(countries, othersArray);

Ramda REPL

each of the helper functions contain a specialization that makes them difficult to reuse and ends up feeling unnatural in a functional program — Mulan, May 08 '19 at 16:15

score 1 · Answer 7 · answered May 08 '19 at 14:15

I would group by the country, merge each country group to a single object, while summing the value, sort, split to two arrays [highest 4] and [others], merge others to a single object, and concat with the highest 4.

const { pipe, groupBy, prop, values, map, converge, merge, head, pluck, sum, objOf, sort, descend, splitAt, concat, last, of, assoc } = R

const sumProp = key => pipe(pluck(key), sum, objOf(key))

const combineProp = key => converge(merge, [head, sumProp(key)])

const getTop5 = pipe(
  groupBy(prop('country')),
  values, // convert to array of country arrays
  map(combineProp('value')), // merge each sub array to a single object
  sort(descend(prop('value'))), // sort descebdubg by the value property
  splitAt(4), // split to two arrays [4 highest][the rest]
  converge(concat, [ // combine the highest and the others object
    head,
    // combine the rest to the others object wrapped in an array
    pipe(last, combineProp('value'), assoc('country', 'others'), of)
  ])
)

const countries = [{ country: 'France', value: 100 }, { country: 'France', value: 100 }, { country: 'Romania', value: 500 }, { country: 'England', value: 400 }, { country: 'England', value: 400 }, { country: 'Spain', value: 130 }, { country: 'Albania', value: 4 }, { country: 'Hungary', value: 3 }]

const result = getTop5(countries)

console.log(result)

<script src="https://cdnjs.cloudflare.com/ajax/libs/ramda/0.26.1/ramda.js"></script>

score 1 · Answer 8 · answered May 08 '19 at 15:04

I would probably do something like this:

const aggregate = R.pipe(
  R.groupBy(R.prop('country')),
  R.toPairs,
  R.map(
    R.applySpec({ 
      country: R.head, 
      value: R.pipe(R.last, R.pluck('value'), R.sum),
    }),
  ),
  R.sort(R.descend(R.prop('value'))),
  R.splitAt(4),
  R.over(
    R.lensIndex(1), 
    R.applySpec({ 
      country: R.always('Others'), 
      value: R.pipe(R.pluck('value'), R.sum),
    }),
  ),
  R.unnest,
);

const data = [
  { country: 'France', value: 100 },
  { country: 'France', value: 100 },
  { country: 'Romania', value: 500 },
  { country: 'England', value: 400 },
  { country: 'England', value: 400 },
  { country: 'Spain', value: 130 },
  { country: 'Albania', value: 4 },
  { country: 'Hungary', value: 3 }
];

console.log('result', aggregate(data));

<script src="https://cdnjs.cloudflare.com/ajax/libs/ramda/0.26.1/ramda.js"></script>

Chris Vouga · Answer 9 · 2019-06-14T05:58:21.340

Here's two solutions

I think the second is easier to understand even though it's longer

The function "mergeAllWithKeyBy" combines the functionality of "R.mergeAll", "R.mergeWithKey", and "R.groupBy".

const mergeAllWithKeyBy = R.curry((mergeFn, keyFn, objs) =>
  R.values(R.reduceBy(R.mergeWithKey(mergeFn), {}, keyFn, objs)))

const addValue = (k, l, r) => 
  k === 'value' ? l + r : r

const getTop = 
  R.pipe(
    mergeAllWithKeyBy(addValue, R.prop('country')),
    R.sort(R.descend(R.prop('value'))),
    R.splitAt(4),
    R.adjust(-1, R.map(R.assoc('country', 'Others'))),
    R.unnest,
    mergeAllWithKeyBy(addValue, R.prop('country')),
  )
  
const data = [
  { country: 'France', value: 100 },
  { country: 'France', value: 100 },
  { country: 'Romania', value: 500 },
  { country: 'England', value: 400 },
  { country: 'England', value: 400 },
  { country: 'Spain', value: 130 },
  { country: 'Albania', value: 4 },
  { country: 'Hungary', value: 3 }
]

console.log(getTop(data))

<script src="//cdn.jsdelivr.net/npm/ramda@latest/dist/ramda.min.js"></script>

const getTop = (data) => {
  const getCountryValue =
    R.prop(R.__, R.reduceBy((y, x) => y + x.value, 0, R.prop('country'), data))
    
  const countries = 
    R.uniq(R.pluck('country', data))
  
  const [topCounties, bottomCountries] = 
    R.splitAt(4, R.sort(R.descend(getCountryValue), countries))
  
  const others = {
    country: 'Others', 
    value: R.sum(R.map(getCountryValue, bottomCountries))
  }
  
  const top =
    R.map(R.applySpec({country: R.identity, value: getCountryValue}), topCounties)
  
  return R.append(others, top)
}

const data = [
  { country: 'France', value: 100 },
  { country: 'France', value: 100 },
  { country: 'Romania', value: 500 },
  { country: 'England', value: 400 },
  { country: 'England', value: 400 },
  { country: 'Spain', value: 130 },
  { country: 'Albania', value: 4 },
  { country: 'Hungary', value: 3 }
]

console.log(getTop(data))

<script src="//cdn.jsdelivr.net/npm/ramda@latest/dist/ramda.min.js"></script>

creating top 5 aggregation with ramdajs

9 Answers9

Update

Here's two solutions