Java - Concise comprehensions-like functionality using function overloads

Question

It's no secrete that one of the most common rants about java is that it's so damn verbose.. so I decided one day to toy with writing some simple code to help myself with this problem. I wanted to have something like python's comprehension expressions. so I focused on capturing the main aspect which I feel makes them useful, which is that they condense 4 logical operations into a single expression -

1. creating a new container to hold the result
2. iterating over every element in the original container
3. filtering (if you want)
4. potentially applying some function before the elements are inserted to the result container.

so if in java you would write

Set<Integer> numbers = Set.of(1,2,3,4,5,6,7,8); // assuming you have this set of numbers

// Pre java 8
ArrayList<Integer> evensSquared = new ArrayList<>();
for (Integer i: numbers){
    if (i % 2 == 0)
        evens.add(i*i);
}


// Post java 8
List<Integer> evensSquared = numbers.stream().filter(i -> i % 2 == 0).map(x -> x * x).collect(Collectors.toList());

in python you could just

evens_squared = {x * x for x in numbers if x % 2 == 0}

now, to be honest this simple example is not really that horrible. And even with some more realistic examples that span a few more lines the big problem is not that it would take me another 20 seconds to type the code in java, rather that in general the language feels so inelegant that after writing a file or few I'd often be looking at the code displeased, spending a few more hours refactoring just to make it feel less clunky. And this can be a real time spender.

So I've written some code to try my version of comprehensions

import java.util.ArrayList;
import java.util.Collection;
import java.util.Iterator;
import java.util.Objects;
import java.util.function.Function;
import java.util.function.Predicate;

// naive implementation, can be made better of course

public abstract class Comprehension<T,K, TCollection extends Collection<T>, KCollection extends Collection<K>>{

        protected abstract TCollection getSinkAsTCollection();

        protected abstract KCollection getSinkAsKCollection();

        TCollection comprehend(Iterator<T> source){
            return collectionComprehension(source, x -> true, Function.identity(), getSinkAsTCollection());
        }

        TCollection comprehend(Iterator<T> source, Predicate<T> filter){
            return collectionComprehension(source, filter, Function.identity(), getSinkAsTCollection());
        }

        KCollection comprehend(Iterator<T> source, Predicate<T> filter, Function<T,K> action){
            return collectionComprehension(source, filter, action, getSinkAsKCollection());
        }

    public static <T,K,U extends Collection<K>> U collectionComprehension(Iterator<T> source, Predicate<T> filter, Function<T,K> action, U sink) {
        Objects.requireNonNull(source);
        Objects.requireNonNull(action);
        Objects.requireNonNull(sink);

        ArrayList<K> transformedElements = new ArrayList<>(); // accumulating elements into this list first to avoid a ConcurrentModificationException in case source and sink are backed by the same structure

        filter = filter == null? x->true: filter;
        for (Iterator<T> it = source; it.hasNext(); ) {
            T t = it.next();
            if (filter.test(t))
                transformedElements.add(action.apply(t));
        }

        sink.addAll(transformedElements);
        return sink;
    }

    public static <T,K,U extends Collection<K>> U collectionComprehension(Iterable<T> source, Predicate<T> filter, Function<T,K> action, U sink) {
            return collectionComprehension(source.iterator(), filter, action, sink);
    }

}

and two examples for concrete comprehension classes:

package Conversions;

import java.util.HashSet;

public class SetComprehension<T,K> extends Comprehension<T,K, HashSet<T>,HashSet<K>> {

    @Override
    protected HashSet<T> getSinkAsTCollection() {
        return new HashSet<>();
    }

    @Override
    protected HashSet<K> getSinkAsKCollection() {
        return new HashSet<>();
    }
    
}

import java.util.ArrayList;

public class ListComprehension<T,K> extends Comprehension<T,K, ArrayList<T>,ArrayList<K>> {

    @Override
    protected ArrayList<T> getSinkAsTCollection() {
        return new ArrayList<>();
    }

    @Override
    protected ArrayList<K> getSinkAsKCollection() {
        return new ArrayList<>();
    }
}

At this point you could write (in keeping with the previous example)

List<Integer> evensSquared = new ListComprehension<Integer,Integer>().comprehend(numbers.iterator(), x -> x % 2 == 0, x -> x * x);

but compare this with the stream version, this new take is even worse!..

List<Integer> evensSquared = numbers.stream().filter(num -> num % 2 == 0).map(x -> x * x).collect(Collectors.toList());

To shorten this to anything idiomatic you'd need to supply static methods to alleviate the need for the class instantiation ceremony, iterator() and type qualifiers. so you start adding some static methods to the ListComprehension class...

    public static <T> List<T> list(Iterable<T> source) {
        return new ListComprehension<T,T>().comprehend(source.iterator());
    }

    public static <T> List<T> list(Iterable<T> source, Predicate<T> filter) {
        return new ListComprehension<T,T>().comprehend(source.iterator(), filter);
    }

    public static <T,K> List<K> list(Iterable<T> source, Predicate<T> filter, Function<T,K> action) {
        return new ListComprehension<T,K>().comprehend(source.iterator(),filter,action);
    }

and now it works great! nice and short!

List<Integer> evensSquared = list(integers, num -> num % 2 == 0, x -> x * x);

However, now you need to handle a different problem - if you want to support even just the basic java containers as input you would need to write no less than a mind boggling 45(!!!) such overloads. How come? because java has fragmented itself into no less than 6!! different container types, with no common denominator to any of them:

1. Iterable
2. Iterator (which is not iterable)
3. Stream (which is not iterable)
4. BaseStream: IntStream, DoubleStream, LongStream (all of which are neither iterable nor a stream)
5. Object Array (again, is none of the above)
6. Primitive Array - with 8 types of primitives.. and not a subtype of any other option

So in total you have 15 types of container * 3 signatures = 45 static functions.. for each type of Comprehension you would want to make.

So to sum, my questions are:

1. what do you think about the api? I obviously recon why java haven't done that itself in it's streams as it would explode the Streams api, but it's just really handy to have a concise way of converting any container to the common ones without the usual ceremony.
2. is there any common denominator to at least some of the above types that I missed, or any other method I can use to reduce the number of overloads you would need to cover the conversions between at least some of those types?
3. Are there any additional java built containers that I missed and aren't covered by any of the above signatures?
4. Are there any commonly used java libraries with container types that aren't covered by any of the above signatures? for example I think google's Guava collections are Iterable so they should be covered, but vavr's collections are rather Traversable so would need additional static overrides
5. have any library already done this painstaking job? feels like a job for code generators

Thanks a bunch

P.S - Since each input type has 3 signatures I could of course drop the partial ones and leave only the overloads that have all parameters supplied(i.e send params such as filter=x->true and action=x->x when no filter/mapping is needed).. but the whole point was making these expressions as terse as possible

Answer 1

The problem of your approach is not the amount of possible inputs, but that all that you have yet achieved, is a compact way to produce a list. If you want something other than a list, your list(…) method is useless.

So you have two choices

1. Create a new method for each imaginable result and limit the use cases to what you have imagined. Or

2. Create an abstraction that allows to be expanded to produce results not hard-coded into your solution. Then, realize…

…that you have just re-invented the Stream API.

As you already noticed, there are different sources to be integrated

• Collections: collection.stream()

• Arrays, of objects or int, long, or double: Arrays.stream(array)

• Other primitive arrays: IntStream.range(0, array.length).map(ix -> array[ix]) (resp. mapToDouble or mapToObj)

• A String (chars() or codePoints()) or a BufferedReader (lines()) or regex matches (matcher.results()),
  when you understand that getting a stream is the solution, rather than seeing a stream as a problem (another source to be integrated into your Stream reinvention).

• Obviously, there is no need to integrate a Stream into a Stream and an Iterator is not a source but a mediator between the source and the target. However, solutions exist, see How to convert an Iterator to a Stream? and Convert Iterable to Stream using Java 8 JDK

• The point of an abstraction is, that new sources can be integrated by independent developers, like with java.util.stream with ResultSet

Then, you might want to support producing a result other than a list and end up at adding a new method for every imaginable result (as said above) or reinvent the Collector abstraction to support two dozen combinable built-in results and be extensible at the same time. Plus the option to use other terminal operations for counting, testing, plain reduction, etc.

But since producing a list has been recognized as a common case worth a dedicated method, you can write

List<Integer> evensSquared = numbers.stream()
    .filter(i -> i % 2 == 0).map(x -> x * x).toList();

in recent Java versions, which can be written only slightly more compact with your approach, because you’re hardcoding both, the result and the intermediate operations, only supporting filtering-followed-by-mapping, unless you’re adding another overload to the method.

I can easily use the Stream API to write

List<String> squaresWithTwoDigits = numbers.stream()
    .map(x -> String.valueOf(x * x)).filter(s -> s.length() == 2).toList();

instead, without waiting for the Stream API authors to provide a suitable method for “do map, followed by filter”. There’s an infinite number of combinations of intermediate operations, but only a finite number of methods you can add to your API.

---

To sum it up, you can always create a slightly more compact API by reducing the flexibility. But that’s not a proof that the far more flexible API was too verbose.

If you really think that your specific use case deserve a more compact method, you can implement your utility method(s) using the Stream API, instead of reinventing the Stream API…

public static void main(String args[]) {
    Set<Integer> numbers = Set.of(1,2,3,4,5,6,7,8);
    List<Integer> evensSquared = list(numbers, i -> i % 2 == 0, x -> x * x);
    System.out.println(evensSquared);
}
public static <T,K> List<K> list(
    Collection<T> source, Predicate<T> filter, Function<T,K> action) {

    return source.stream().filter(filter).map(action).toList();
}