Aggregate List of objects in Java

Question

Do we have any aggregator function in Java to perform the below aggregation?

Person {
    String name;
    String subject;
    String department;
    Long mark1;
    Long mark2;
    Long mark3;
}

List contains data as below.

Name    |Subject    |Department |Mark1  |Mark2  |Mark3
--------|-----------|-----------|-------|-------|-----
Clark   |English    |DEP1       |7      |8      |6
Michel  |English    |DEP1       |6      |4      |7
Dave    |Maths      |DEP2       |3      |5      |6
Mario   |Maths      |DEP1       |9      |7      |8

The aggregation criteria is Subject & Dep. The resultant object needs to be

Subject     |Department |Mark1  |Mark2  |Mark3
----------- |-----------|-------|-------|-----
English     |DEP1       |13     |12     |13
Maths       |DEP2       |3      |5      |6
Maths       |DEP1       |9      |7      |8

This aggregation can be achieved by manually iterating through the list and create an aggregated list. Example as below.

private static List<Person> getGrouped(List<Person> origList) {
    Map<String, Person> grpMap = new HashMap<String, Person>();

    for (Person person : origList) {
        String key = person.getDepartment() + person.getSubject();
        if (grpMap.containsKey(key)) {
            Person grpdPerson = grpMap.get(key);
            grpdPerson.setMark1(grpdPerson.getMark1() + person.getMark1());
            grpdPerson.setMark2(grpdPerson.getMark2() + person.getMark2());
            grpdPerson.setMark3(grpdPerson.getMark3() + person.getMark3());
        } else {
            grpMap.put(key, person);
        }
    }
    return new ArrayList<Person>(grpMap.values());
}

But is there any aggregation function or feature of Java 8 which we can leverage?

Look into http://stackoverflow.com/questions/21020562/aggregate-functions-over-a-list-in-java it has an example of doing a similar grouping using Stream.collect — Jaiprakash, Jun 23 '16 at 07:08
@Kayaman The grouping function returns Map. But this will require List go grouped objects. — Swadeesh, Jun 23 '16 at 07:32
@Jaiprakash I checked those links. My class cannot have a hash code as aggregation can be based on different attributes. — Swadeesh, Jun 23 '16 at 07:32

Jaiprakash · Answer 1 · 2016-06-23T10:44:19.303

You can use reduction. Sample to aggregate mark1 is as follows.

public class Test {

    static class Person {
        Person(String name, String subject, String department, Long mark1, Long mark2, Long mark3) {
            this.name = name;
            this.subject = subject;
            this.department = department;
            this.mark1 = mark1;
            this.mark2 = mark2;
            this.mark3= mark3;
        }
            String name;
            String subject;
            String department;
            Long mark1;
            Long mark2;
            Long mark3;

            String group() {
                return subject+department;
            }

            Long getMark1() {
                return mark1;
            }
    }

      public static void main(String[] args)
      {
        List<Person> list = new ArrayList<Test.Person>();
        list.add(new Test.Person("Clark","English","DEP1",7l,8l,6l));
        list.add(new Test.Person("Michel","English","DEP1",6l,4l,7l));
        list.add(new Test.Person("Dave","Maths","DEP2",3l,5l,6l));
        list.add(new Test.Person("Mario","Maths","DEP1",9l,7l,8l));

        Map<String, Long> groups = list.stream().collect(Collectors.groupingBy(Person::group, Collectors.reducing(
                    0l, Person::getMark1, Long::sum)));

        //Or alternatively as suggested by Holger 
        Map<String, Long> groupsNew = list.stream().collect(Collectors.groupingBy(Person::group, Collectors.summingLong(Person::getMark1)));

        System.out.println(groups);

      }

}

Still looking into generating the output via a single functions. Will update once completed.

Alexis C. · Accepted Answer · 2016-06-23T12:23:45.150

Using standards collectors in the JDK, you can do it like this (assuming the creation of a Tuple3<E1, E2, E3> class):

Map<String, Map<String, Tuple3<Long, Long, Long>>> res =
    persons.stream().collect(groupingBy(p -> p.subject,
                                        groupingBy(p -> p.department,
                                                   reducing(new Tuple3<>(0L, 0L, 0L), 
                                                            p -> new Tuple3<>(p.mark1, p.mark2, p.mark3), 
                                                            (t1, t2) -> new Tuple3<>(t1.e1 + t2.e1, t1.e2 + t2.e2, t1.e3 + t2.e3)))));

This will first group the elements by their subject, then by department and reduces the resulting values in the second map by summing their marks.

Running it on the list of persons you have in your example, you'll get as output:

Maths => DEP2 => (3, 5, 6)
Maths => DEP1 => (9, 7, 8)
English => DEP1 => (13, 12, 13)

In this case you may also want to use another variant using the toMap collector. The logic remains the same, the function to map the values will create a map containing the department as a key, and the grade of the student as a value. The merge function will be in charge to add or update the mappings.

Map<String, Map<String, Tuple3<Long, Long, Long>>> res3 =
        persons.stream()
               .collect(toMap(p -> p.subject,
                              p -> {
                                  Map<String, Tuple3<Long, Long, Long>> value = new HashMap<>();
                                  value.put(p.department, new Tuple3<>(p.mark1, p.mark2, p.mark3));
                                  return value;
                              },
                              (v1, v2) -> {
                                   v2.forEach((k, v) -> v1.merge(k, v, (t1, t2) -> new Tuple3<>(t1.e1 + t2.e1, t1.e2 + t2.e2, t1.e3 + t2.e3)));
                                   return v1;
                              }
               ));

Of course you can question yourself about the "beauty" of these solutions, maybe you want to introduce a custom collector or custom classes to make the intent more clear.

score 1 · Answer 3 · edited May 23 '17 at 12:16

Using the approach from Group by multiple field names in java 8 with a custom key class, my suggestion is this:

    Map<DepSubject, Grades> map = persons.stream().
            collect(Collectors.groupingBy(x -> new DepSubject(x.department, x.subject),
            Collectors.reducing(
                    new Grades(0, 0, 0),
                    y -> new Grades(y.mark1, y.mark2, y.mark3),
                    (x, y) -> new Grades(x.m1 + y.m1, x.m2 + y.m2, x.m3 + y.m3)
            )));

The DepSubject defines equals and hashCode. This way the original class has not to be changed, and if multiple grouping criteria are needed, multiple classes can be used. Unfortunately, this can be quite verbose in Java, since you need a class with equals, hashCode, (getters, setters). Actually, in my opinion, getters and setters could also be omitted, if the class is only used in one place for grouping.

class DepSubject{ 

    String department;
    String subject;

    public DepSubject(String department, String subject) {
        this.department = department;
        this.subject = subject;
    }

    public String getDepartment() {
        return department;
    }
    // equals,hashCode must also be defined for this to work, omitted for brevity
    }

It is also possible to collect the results into List. This way, the custom classes DepSubject and Grades are just used for intermediate operations:

    List<Person> list = persons.stream().
            collect(Collectors.collectingAndThen(
                    Collectors.groupingBy(x -> new DepSubject(x.department, x.subject),
                            Collectors.reducing(
                                    new Grades(0, 0, 0),
                                    y -> new Grades(y.mark1, y.mark2, y.mark3),
                                    (x, y) -> new Grades(x.m1 + y.m1, x.m2 + y.m2, x.m3 + y.m3)
                            )),
                    map -> map.entrySet().stream()
                              .map(e -> new Person(null, e.getKey().subject, e.getKey().department, e.getValue().m1, e.getValue().m2, e.getValue().m3))
                              .collect(Collectors.toList())
            ));

You could also extract the groupingBy logic into a function:

private static <T> List<Person> groupBy(List<Person> persons, Function<Person,T> function, BiFunction<T,Grades,Person> biFunction) {
    return persons.stream().
            collect(Collectors.collectingAndThen(
                    Collectors.groupingBy(function,
                            Collectors.reducing(
                                    new Grades(0, 0, 0),
                                    y -> new Grades(y.mark1, y.mark2, y.mark3),
                                    (x, y) -> new Grades(x.m1 + y.m1, x.m2 + y.m2, x.m3 + y.m3)
                            )),
                    map -> map.entrySet().stream()
                              .map(e -> biFunction.apply(e.getKey(),e.getValue()))
                              .collect(Collectors.toList())
            ));
}

This way, you can group your Persons this way:

    List<Person> list = groupBy(persons,
            x -> new DepSubject(x.department, x.subject),
            (depSubject,grades) -> new Person(null, depSubject.subject, depSubject.department, grades.m1, grades.m2, grades.m3));

If you want to group your object by subject only, you could just do:

    List<Person> list2 = groupBy(persons,
            Person::getSubject,
            (subject,grades) -> new Person(null,subject, null, grades.m1, grades.m2, grades.m3));

Thanks for the answer, but I'm stuck with this Class structure. I will not be able to split them. Also, I will not be able to define equals and hashcode, as I might need grouping based on multiple keys. — Swadeesh, Jun 23 '16 at 09:11
@Swad: you only need that classes for intermediate operations. you can start with `List` and map the results back to a `List` If you need multiple groupings, you can also use different classes. Unfortunately, this is quite verbose in Java... — user140547, Jun 23 '16 at 09:27

Aggregate List of objects in Java

3 Answers3