Summarize and collect stream elements at same time

Question

Using java streams api, I would like to get SummaryStatistics (min, max , etc.) and collect all the objects into an ArrayList at the same time.

The following gives an exception because the stream is already processed with summaryStatistics() call.

Can these two operations be combined into one operation.

Code example :

IntSummaryStatistics summaryStatistics = statistics.mapToInt(Statistic::getSpread)
        .summaryStatistics();

List<Statistics> statsList = statistics.collect(Collectors.toList());

Thank you.

what is the source of your `statistics` stream? just to know, why do you want it to be collected to a `List` in your result? — Naman, Nov 05 '20 at 15:39

fps · Answer 1 · 2020-11-05T16:40:22.653

Starting from Java 12, there is Collectors.teeing which, according to the docs:

Returns a Collector that is a composite of two downstream collectors. Every element passed to the resulting collector is processed by both downstream collectors, then their results are merged using the specified merge function into the final result.

So, in your case, you could use it this wway:

Result result = statistics.collect(
    Collectors.teeing(
        Collectors.summarizingInt(Statistic::getSpread),
        Collectors.toList(),
        Result::new));

Where Result is as follows:

public class Result {

    private final IntSummaryStatistics stats;

    private final List<Statistic> list;

    Result(IntSummaryStatistics stats, List<Statistic> list) {
        this.stats = stats;
        this.list = list;
    }

    // getters
}

Note that if you are on Java 14+, you can use a record for the result, instead of a class:

public record Result(IntSummaryStatistics stats, List<Statistic> list) { }

EDIT: If you are not on Java 12+ yet, you might use this code to create a utility teeing method:

public static <T, R1, R2, R> Collector<T, ?, R> teeing(
  Collector<? super T, ?, R1> downstream1,
  Collector<? super T, ?, R2> downstream2,
  BiFunction<? super R1, ? super R2, R> merger) {
    return teeing0(downstream1, downstream2, merger);
}

private static <T, A1, A2, R1, R2, R> Collector<T, ?, R> teeing0(
  Collector<? super T, A1, R1> downstream1,
  Collector<? super T, A2, R2> downstream2,
  BiFunction<? super R1, ? super R2, R> merger) {
    Objects.requireNonNull(downstream1, "downstream1");
    Objects.requireNonNull(downstream2, "downstream2");
    Objects.requireNonNull(merger, "merger");

    Supplier<A1> c1Supplier = Objects.requireNonNull(
      downstream1.supplier(), "downstream1 supplier");
    Supplier<A2> c2Supplier = Objects.requireNonNull(
      downstream2.supplier(), "downstream2 supplier");
    BiConsumer<A1, ? super T> c1Accumulator = Objects.requireNonNull(
      downstream1.accumulator(), "downstream1 accumulator");
    BiConsumer<A2, ? super T> c2Accumulator = Objects.requireNonNull(
      downstream2.accumulator(), "downstream2 accumulator");
    BinaryOperator<A1> c1Combiner = Objects.requireNonNull(
      downstream1.combiner(), "downstream1 combiner");
    BinaryOperator<A2> c2Combiner = Objects.requireNonNull(
      downstream2.combiner(), "downstream2 combiner");
    Function<A1, R1> c1Finisher = Objects.requireNonNull(
      downstream1.finisher(), "downstream1 finisher");
    Function<A2, R2> c2Finisher = Objects.requireNonNull(
      downstream2.finisher(), "downstream2 finisher");

    Set<Collector.Characteristics> characteristics;
    // Characteristics left as an exercise to the reader :)

    class PairBox {
        A1 left = c1Supplier.get();
        A2 right = c2Supplier.get();

        void add(T t) {
            c1Accumulator.accept(left, t);
            c2Accumulator.accept(right, t);
        }

        PairBox combine(PairBox other) {
            left = c1Combiner.apply(left, other.left);
            right = c2Combiner.apply(right, other.right);
            return this;
        }

        R get() {
            R1 r1 = c1Finisher.apply(left);
            R2 r2 = c2Finisher.apply(right);
            return merger.apply(r1, r2);
        }
    }

    return new CollectorImpl<>(PairBox::new, PairBox::add, 
      PairBox::combine, PairBox::get, characteristics);
}

I should have specified that I'm on java 11 but thank you for the info... this is what I was hoping for. — Albert 2Rocks, Nov 05 '20 at 15:58
@Albert2Rocks I'll add an edit with the `Collectors.teeing` method's code — fps, Nov 05 '20 at 16:28

score 1 · Answer 2 · answered Nov 05 '20 at 13:56

1

The simplest solution is to just collect to a List and do your follow-up operations on the List.

List<Statistics> statsList = statistics.collect(Collectors.toList());

IntSummaryStatistics summaryStatistics = statsList.stream()
        .mapToInt(Statistic::getSpread)
        .summaryStatistics();

answered Nov 05 '20 at 13:56

MikeFHay

8,562
4
31
52

Yes , it is simple. What bothers me is to create the list which means to go through the whole stream. Then to get the statistics you have to go through the stream again. – Albert 2Rocks Nov 05 '20 at 14:26
Why should that bother you? – MikeFHay Nov 05 '20 at 14:29
I have to handle VERY large data sets. – Albert 2Rocks Nov 05 '20 at 14:35
Can't be that large if you're collecting them into an in-memory List. – MikeFHay Nov 05 '20 at 14:41
1

Also iterating the stream once and doing 2 pieces of work per item is not necessarily more efficient than iterating the stream twice. – MikeFHay Nov 05 '20 at 14:43

score 0 · Answer 3 · answered Nov 05 '20 at 12:20

It's possible if, for example, you add the elements to the List as a side effect of the mapToInt function:

List<Statistics> statsList = new ArrayList<>();
IntSummaryStatistics summaryStatistics = 
    statistics.mapToInt(s -> {statsList.add(s); return s.getSpread();})
              .summaryStatistics();

However, I suggest to first collect the elements into a List and then compute the statistics using a new Stream:

List<Statistics> statsList = statistics.collect(Collectors.toList());
IntSummaryStatistics summaryStatistics = statsList.stream()
                                                  .mapToInt(Statistic::getSpread)
                                                  .summaryStatistics();

Summarize and collect stream elements at same time

3 Answers3