How to get the difference between the last and first element's value using Java8 stream?

Question

The problem at hand is to return the change between the first and last known measurement in a certain interval from a series of timestamp/value data. Also, I'd like to learn to use Java 8 Streams, so I'm trying to see if and how this could solve the problem.

A sample of the data:

DateTime,Value
...
1470012671,618.59
1470012912,618.62
1470013212,618.65
1470013512,618.68
1470013632,618.69
1470013900,618.71
...

Example input:

startMillis: 1470012800
endMillis: 1470013800

Expected answer (I choose the 'inner key values' when the start and end time are not present (see bonus question below)):

618.69 - 618.62 = 0.07

The code I have so far:

double amountKiloWattHours = 0;
long startMillis = startingTime.toEpochSecond();
long endMillis = startMillis + PERIOD_LENGTH_MILLIS;
try (Stream<String> stream = Files.lines(Paths.get(fileName))) {
        // Currently, only returns all pairs within range...
    Stream<Pair> pairs = stream
        .skip(1)
        .map(p -> {
            return new Pair(p);
        })
        .filter(pair -> {
            return (pair.getMillis() > startMillis) && (pair.getMillis() < endMillis);
       });
} catch (Exception e) {
    // TODO specify and handle exceptions...
}

.

public class Pair {
    @Getter
    private final long millis;
    @Getter
    private final double kWhs;

    public Pair(String input) {
        String[] parts = input.split(",");
        this.millis = Long.parseLong(parts[0]);
        this.kWhs = Double.parseDouble(parts[1]);
    }
}

How do I now get the difference between the value of the last and the first pair in the interval?

Bonus question: How do I get the interpolated result where the exact timestamps' value is linearly interpolated between two surrounding values?

Answer 1

There's no ready collector which can find you both maximal and minimal elements, but you can use the pairing collector I wrote in this answer (and which is readily available in my StreamEx library -- see JavaDoc):

Comparator<Pair> cmp = Comparator.comparingLong(Pair::getMillis);

double diff = stream
    .skip(1)
    .map(Pair::new)
    .filter(pair -> (pair.getMillis() > startMillis) && (pair.getMillis() < endMillis))
    .collect(pairing(Collectors.maxBy(cmp), Collectors.minBy(cmp),
      (maxPair, minPair) -> maxPair.get().getKWhs() - minPair.get().getKWhs()));

This will not collect unnecessary data: you will extract only min and max row.

Note: this code assumes that you have at least one Pair (ie. one timestamp/value instance within range, not necessarily both a distinct minimum and maximum timestamp/value) satisfying the condition. If you want to handle specially the case when there are no such pair (for example, return 0), you should modify the code (for example, like this: maxPair.map(Pair::getKWhs).orElse(0d) - minPair.map(Pair::getKWhs).orElse(0d)).

Answer 2

It's still handy to filter out the relevant "in-range" records using streams:

<T, U extends Comparable<U>> Predicate<T> inRange(U start, U end, Function<T, U> get) {
    return t -> Optional.of(t)
            .map(get)
            .filter(u -> start.compareTo(u) < 0 && 0 < end.compareTo(u))
            .isPresent();
}

List<Double> list = stream.skip(1)
        .map(Pair::new)
        .filter(inRange(startMillis, endMillis, Pair::getMillis))
        .map(Pair::getKWhs)
        .collect(Collectors.toList())

double diff = list.get(list.size() - 1) - list.get(0);

But then you still need to hand-pick the first and the last element from the list to substract them.

Answer 3

These are meter readings so I will assume both millis and kWhs are monotonically increasing. I would use the BiFunction form of the reduce stream function. I would create an accumulator class that accumulates the first and last seen values (i.e., a Pair). That class would look something like this:

private class Ac {
    private final Pair first;
    private final Pair last;

    public Ac() {
        this.first = null;
        this.last = null;
    }

    public Ac(Ac prev, Ac next) {
        this.first = prev.first != null ? prev.first : next.first;
        this.last = next.last;
    }

    public Ac(Ac prev, Pair next) {
        if (prev.first != null) {
            this.first = prev.first;
            this.last = next;
        } else {
            this.first = next;
            this.last = null;
        }
    }

    public double diff() {
        return first != null && last != null ? last.getkWhs() - first.getkWhs() : 0;
    }
}

Supporting interpolation requires a little more work, but it's just more of the same.

The stream part of the code would look like the following:

try (Stream<String> stream = Files.lines(Paths.get(fileName))) {
    Ac result = stream
                .skip(1)
                .map(x -> new Pair(x))
                .filter(y -> (y.getMillis() > startMillis) && (y.getMillis() < endMillis))
                .reduce(new Ac(), (u, t) -> new Ac(u, t), (u1, u2) -> new Ac(u1, u2));
    amountKiloWattHours = result.diff();

There are a lot of pairwise processing problems that can be solved with non-trivial accumulators, but I would be wary of introducing this code unless I could justify the complexity. Am I enhancing reusability or not? Am I improving readability or not? Streams and lambda expressions are great, but they are not always the right solution.