608

I am looking for a concise way to convert an Iterator to a Stream or more specifically to "view" the iterator as a stream.

For performance reason, I would like to avoid a copy of the iterator in a new list:

Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();
Collection<String> copyList = new ArrayList<String>();
sourceIterator.forEachRemaining(copyList::add);
Stream<String> targetStream = copyList.stream();

Based on the some suggestions in the comments, I have also tried to use Stream.generate:

public static void main(String[] args) throws Exception {
    Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();
    Stream<String> targetStream = Stream.generate(sourceIterator::next);
    targetStream.forEach(System.out::println);
}

However, I get a NoSuchElementException (since there is no invocation of hasNext)

Exception in thread "main" java.util.NoSuchElementException
    at java.util.AbstractList$Itr.next(AbstractList.java:364)
    at Main$$Lambda$1/1175962212.get(Unknown Source)
    at java.util.stream.StreamSpliterators$InfiniteSupplyingSpliterator$OfRef.tryAdvance(StreamSpliterators.java:1351)
    at java.util.Spliterator.forEachRemaining(Spliterator.java:326)
    at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
    at Main.main(Main.java:20)

I have looked at StreamSupport and Collections but I didn't find anything.

gontard
  • 28,720
  • 11
  • 94
  • 117

11 Answers11

690

One way is to create a Spliterator from the Iterator and use that as a basis for your stream:

Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();
Stream<String> targetStream = StreamSupport.stream(
          Spliterators.spliteratorUnknownSize(sourceIterator, Spliterator.ORDERED),
          false);

An alternative which is maybe more readable is to use an Iterable - and creating an Iterable from an Iterator is very easy with lambdas because Iterable is a functional interface:

Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();

Iterable<String> iterable = () -> sourceIterator;
Stream<String> targetStream = StreamSupport.stream(iterable.spliterator(), false);
Neuron
  • 5,141
  • 5
  • 38
  • 59
assylias
  • 321,522
  • 82
  • 660
  • 783
  • Thanks, is it really a `view` of the iterator or a copy ? – gontard Jul 01 '14 at 13:35
  • 34
    Stream are lazy: the code is only linking the Stream to the Iterator but the actual iteration won't happen until a terminal operation. If you use the iterator in the meantime you won't get the expected result. For example you can introduce a `sourceIterator.next()` before using the stream and you will see the effect (the first item will not be seen by the Stream). – assylias Jul 01 '14 at 13:36
  • Would `Spliterator.ORDERED` also be a valid charasteric for any iterator? – skiwi Jul 01 '14 at 13:39
  • @skiwi yes you are right - and actually SIZED is removed by the `spliteratorUnknownSize` method. I have amended. – assylias Jul 01 '14 at 13:43
  • 12
    @assylias, yeah it is really nice ! Maybe you could explains for future readers this quite magic line `Iterable iterable = () -> sourceIterator;`. I have to admit that it took me some time to understand. – gontard Jul 02 '14 at 09:55
  • @gontard The line is still magical to me. Can you tell me what you found? Is it a `Supplier>`? – Jin Kwon Jan 13 '15 at 09:03
  • 9
    I should tell what I found. `Iterable` is a `FunctionalInterface` which has only one abstract method `iterator()`. So `() -> sourceIterator` is a lambda expression instantiating an `Iterable` instance as an anonymous implementation. – Jin Kwon Jan 13 '15 at 09:08
  • 18
    Again, `() -> sourceIterator;` is a shortened form of `new Iterable<>() { @Override public Iterator iterator() { return sourceIterator; } }` – Jin Kwon Jan 13 '15 at 12:23
  • 10
    @JinKwon It is not really a shortened form of an anonymous class (there are a few subtle differences, such as scope and how it is compiled) but it does behave similarly in this case. – assylias Jan 13 '15 at 13:53
  • 5
    The tricky bit with () -> sourceIterator is that if it's called more than once, it'll return the *same* iterator each time. This leads to unexpected results. So you're depending on the implementation of StreamSupport.stream() and spliterator(_, false) here. – flup May 23 '16 at 09:52
  • @flup the stream could be use only once, so it is not a problem. – gontard Aug 11 '16 at 12:16
  • 4
    To prevent the problems above with reusing smae iterator, use the following: `Iterable iterable = Arrays.asList("A", "B", "C")::iterator; Stream targetStream = StreamSupport.stream(iterable.spliterator(), false);` – Uwe Schindler Oct 26 '16 at 12:04
  • @Stephan I don't think it works (it will throw an exception at the end of the iterator). – assylias Jun 11 '17 at 18:24
  • @assylias I tried with an ArrayList iterator and it worked. However I passed Arraylist.size() method to limit() method. – Stephan Jun 11 '17 at 19:22
  • @Stephan if you receive the iterator from a collection it works indeed, but you could have an iterator with an unknown​ size, for example a database cursor like in https://stackoverflow.com/a/32232173/829571 or https://stackoverflow.com/a/38990744/829571 – assylias Jun 11 '17 at 23:36
  • Lists.newArrayList(your iterator here).stream() – Jacques Koorts Nov 17 '17 at 11:27
  • What happens when you go parallel in the stream? Must the iterator not be threadsafe to avoid concurrency issues? – mmirwaldt Mar 18 '18 at 17:37
  • 1
    @mmirwaldt You should check Spliterator's javadoc. In particular "*Despite their obvious utility in parallel algorithms, spliterators are not expected to be thread-safe; instead, implementations of parallel algorithms using spliterators should ensure that the spliterator is only used by one thread at a time.*" – assylias Mar 19 '18 at 15:47
  • Seems like Java features get "part way there" and then just stop. This is such an obvious omission from the Stream API! – Josh M. Nov 17 '20 at 12:57
  • @assylias [guava streams](https://github.com/google/guava/blob/ff696deb11d83a59e1befd4a146b96b5e8b8fd3d/guava/src/com/google/common/collect/Streams.java#L83) implementation does not set the `Spliterator.ORDERED` flag. Would this make any difference for the specific solution? – Marinos An Apr 08 '21 at 08:39
  • @MarinosAn in my example the source is a list so ORDERED makes sense, but I guess that if the source if a set for example, it doesn't. – assylias Apr 08 '21 at 11:47
168

Since version 21, Guava library provides Streams.stream(iterator)

It does what @assylias's answer shows.

Sled
  • 18,541
  • 27
  • 119
  • 168
numéro6
  • 3,921
  • 1
  • 19
  • 18
  • 3
    Javadoc: https://static.javadoc.io/com.google.guava/guava/21.0/com/google/common/collect/Streams.html#stream-java.util.Iterator- – Henrik Aasted Sørensen Aug 11 '17 at 06:22
  • Far better to use this consistently until such time as the JDK supports a native one-liner. It will be far simpler to find (hence refactor) this in future than the pure-JDK solutions shown elsewhere. – drekbour Feb 06 '20 at 10:02
  • 7
    This is excellent but… how does Java have native iterators and streams… but no built-in, straightforward way to go from one to the other!? Quite an omission in my opinion. – Dan Lenski Feb 20 '20 at 21:43
109

Great suggestion! Here's my reusable take on it:

public class StreamUtils {

    public static <T> Stream<T> asStream(Iterator<T> sourceIterator) {
        return asStream(sourceIterator, false);
    }

    public static <T> Stream<T> asStream(Iterator<T> sourceIterator, boolean parallel) {
        Iterable<T> iterable = () -> sourceIterator;
        return StreamSupport.stream(iterable.spliterator(), parallel);
    }
}

And usage (make sure to statically import asStream):

List<String> aPrefixedStrings = asStream(sourceIterator)
                .filter(t -> t.startsWith("A"))
                .collect(toList());
Matan
  • 1,437
  • 1
  • 10
  • 6
88

This is possible in Java 9.

Stream.generate(() -> null)
    .takeWhile(x -> iterator.hasNext())
    .map(n -> iterator.next())
    .forEach(System.out::println);
Dónal
  • 185,044
  • 174
  • 569
  • 824
PhilipRoman
  • 1,136
  • 10
  • 20
  • 1
    Simple, efficient and without resorting to subclassing - this should be the accepted answer! – martyglaubitz Sep 16 '19 at 19:56
  • 3
    Unfortunately these don't seem to work with `.parallel()` streams. They also appear a bit slower than going over `Spliterator`, even for sequential use. – Thomas Ahle Nov 26 '19 at 23:30
  • 1
    Also, the first method throws up if the iterator is empty. The second method does work for now, but it does violate the requirement of the functions in map and takeWhile to be stateless, so I'd hesitate to do that in production code. – Dr. Hans-Peter Störr Jan 07 '20 at 07:56
  • Really, this should be an accepted answer. Even though `parallel` might be funky the simpleness is amazing. – Sven Feb 10 '20 at 11:15
  • 5
    You really shouldn't be working with side effects and mutation. – Adam Bickford Sep 17 '20 at 12:25
  • 1
    *While takeWhile() is generally a cheap operation on sequential stream pipelines, it can be quite expensive on ordered parallel pipelines.* Reference: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/stream/Stream.html#takeWhile(java.util.function.Predicate) – dbaltor May 04 '22 at 14:51
11

Create Spliterator from Iterator using Spliterators class contains more than one function for creating spliterator, for example here am using spliteratorUnknownSize which is getting iterator as parameter, then create Stream using StreamSupport

Spliterator<Model> spliterator = Spliterators.spliteratorUnknownSize(
        iterator, Spliterator.NONNULL);
Stream<Model> stream = StreamSupport.stream(spliterator, false);
Tunaki
  • 132,869
  • 46
  • 340
  • 423
Bassem Reda Zohdy
  • 12,662
  • 3
  • 33
  • 39
11
import com.google.common.collect.Streams;

and use Streams.stream(iterator) :

Streams.stream(iterator)
       .map(v-> function(v))
       .collect(Collectors.toList());
theletz
  • 1,713
  • 2
  • 16
  • 22
sneha
  • 107
  • 1
  • 2
1

Edit: Do NOT do this!

(Leaving for archival purposes)

Another way to do this on Java 9+ using Stream::iterate(T, Predicate, UnaryOperator):

Stream.iterate(iterator, Iterator::hasNext, UnaryOperator.identity())
        .map(Iterator::next)
        .forEach(System.out::println);
aksh1618
  • 2,245
  • 18
  • 37
1

1 assylias's solution wrapped in a method:

public static <T> Stream<T> toStream(Iterator<T> iterator) {
    return StreamSupport.stream(((Iterable<T>)() -> iterator).spliterator(), false);
}

2 guava Streams implementation (marked with @Beta):

public static <T> Stream<T> stream(Iterator<T> iterator) {
    return StreamSupport.stream(Spliterators.spliteratorUnknownSize(iterator, 0), false);
}
Marinos An
  • 9,481
  • 6
  • 63
  • 96
0

if iteration size it's known this is possible:

public static void main(String[] args) throws Exception {
    Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();
    Stream<String> targetStream = Stream.generate(sourceIterator::next);
    targetStream.**limit(3)**.forEach(System.out::println);
}
Chavi
  • 1
0

If you are using Spring (Data) then there is a utility class StreamUtils which has two methods:

createStreamFromIterator(Iterator<T> iterator) 

createStreamFromIterator(CloseableIterator<T> iterator)

https://docs.spring.io/spring-data/commons/docs/3.0.0/api/org/springframework/data/util/StreamUtils.html#createStreamFromIterator(java.util.Iterator)

Robert Niestroj
  • 15,299
  • 14
  • 76
  • 119
-5

Use Collections.list(iterator).stream()...