13
Stream.of(a, b, c).parallel().map(Object::toString).iterator();

Is the returned iterator guaranteed to provide the values a, b, c in that order?

I'm aware toArray() and collect() guarantee collections with values in the correct order. Also, I'm not asking how to make a stream from an iterator.

bdkosher
  • 5,753
  • 2
  • 33
  • 40
David Leston
  • 148
  • 8
  • *I'm aware that toArray() and collect() guarantee collections with values in the correct order.* Where did you see that? Many collections don't even have a concept of ordering. – shmosel Jan 10 '18 at 20:26
  • Ok, I found this quote under the [Side-effects](https://docs.oracle.com/javase/8/docs/api/java/util/stream/package-summary.html) section: *`IntStream.range(0,5).parallel().map(x -> x*2).toArray()` must produce `[0, 2, 4, 6, 8]`*. Strange that the documentation isn't clearer about which operations respect encounter order, especially considering the `forEach()`, `forEachOrdered()` distinction. – shmosel Jan 10 '18 at 20:36
  • 1
    Loosely related: my old question about [How does Stream.max() handle equality?](https://stackoverflow.com/q/36711845/2513200) where Holger dug up the following comment by Brian Goetz: ["If the stream is ordered (such as the streams you get from an array or List), it returns the first element that is maximal in the event of multiple maximal elements; only if the stream is unordered is it allowed to pick an arbitrary element."](https://stackoverflow.com/questions/29334404/how-to-force-max-to-return-all-maximum-values-in-a-java-stream/29334774#comment46860038_29334404) – Hulk Jan 12 '18 at 09:23
  • 1
    ...which does not mention sequential vs. parallel, but I took this as a confirmation that all Stream operations that do not explicitly state that they remove the ordering or are explicitly undeterministic are required to maintain encounter order. – Hulk Jan 12 '18 at 09:30

5 Answers5

8

This is an oversight in the specification. If a stream has a defined encounter order, the intent was that its Iterator produce the elements in encounter order. If the stream has no defined encounter order, the Iterator will of course produce the elements in some order, but that order won't be defined.

I've filed bug JDK-8194952 to track the change to the specification.

It looks like others have crawled through enough of the implementation to show that it will indeed produce the elements in encounter order. In addition, our stream tests rely on this property. For example, the test for the toList collector asserts that the elements in the list are present in the same order as they are obtained from the stream's Iterator. So it's probably safe for you to rely on this behavior, even though it isn't formally specified (yet).

Stuart Marks
  • 127,867
  • 37
  • 205
  • 259
  • 2
    Yes, other operations need a clarification too, i.e. even `map` and `filter` don’t specify anything. Or just add an explicit statement that operations maintain the encounter order (if there is one), unless stated otherwise. If we are at it, `max()` and `min()` returning the first one in case of a tie for ordered streams or, e.g. `reduce((a,b)->a)` returning the first element, resp. `reduce((a,b)->b)` returning the last element are also only implicit. And `Stream.concat` being unordered if only one input is unordered, e.g. `concat(range(0, 10), empty())`, is explicit, but a *horrible* decision… – Holger Jan 12 '18 at 10:16
  • Through testing .iterator() appears to make parallel streams into sequential streams which makes this question moot. – David Leston Feb 23 '18 at 16:57
3

The Stream.of method, used to create a stream from otherwise un-associated values, returns an sequential, ordered stream.

Returns a sequential ordered stream whose elements are the specified values.

According to the package Javadocs for java.util.stream, Side Effects section:

IntStream.range(0,5).parallel().map(x -> x*2).toArray() must produce [0, 2, 4, 6, 8]

This implies that parallel() and map() preserve whether the stream is sequential/ordered.

I've traced the implementation of the Stream that Stream.of creates to a class called ReferencePipeline.

@Override
public final Iterator<P_OUT> iterator() {
    return Spliterators.iterator(spliterator());
}

That implementation's iterator() method defers to Spliterator.iterator(), whose code adapts to the Iterator interface by simply relying on the Spliterator's tryAdvance method, and does not change any stream characteristics:

public static<T> Iterator<T> iterator(Spliterator<? extends T> 
    spliterator) {
    Objects.requireNonNull(spliterator);
    class Adapter implements Iterator<T>, Consumer<T> {
        boolean valueReady = false;
        T nextElement;

        @Override
        public void accept(T t) {
            valueReady = true;
            nextElement = t;
        }

        @Override
        public boolean hasNext() {
            if (!valueReady)
                spliterator.tryAdvance(this);
            return valueReady;
        }

        @Override
        public T next() {
            if (!valueReady && !hasNext())
                throw new NoSuchElementException();
            else {
                valueReady = false;
                return nextElement;
            }
        }
    }

    return new Adapter();
}

In conclusion, yes, the order is guaranteed because Stream.of creates a "sequential ordered stream", and none of the operations you use above: parallel, map, or iterator change the characteristics. In fact, iterator uses the underlying Stream Spliterator to iterate over the stream elements.

rgettman
  • 176,041
  • 30
  • 275
  • 357
  • 6
    You can't prove the specification by showing the implementation. – shmosel Jan 10 '18 at 21:07
  • I still think that Javadoc does not "guarantee" order in OP case. Nowhere in Javadoc does it say that `iterator` method will always respect encounter order. – tsolakp Jan 10 '18 at 21:37
  • 2
    The code proves that the `Iterator` does not change the `Spliterator`’s order, so you just changed the question to “Does `spliterator()` on parallel stream guarantee encounter order?” – Holger Jan 11 '18 at 12:45
1

The closest to a guarantee that I have found so far ist the following statement in the package documentaion for java.util.stream:

Except for operations identified as explicitly nondeterministic, such as findAny(), whether a stream executes sequentially or in parallel should not change the result of the computation.

Arguably, iterator() producing an Iterator iterating in a different order would be a "change in the result", just as much as producing a List containing elements in a different order would be for collect().

Hulk
  • 6,399
  • 1
  • 30
  • 52
0

Yes it will. And that is because terminal operations, unless otherwise specified in the documentation (forEach for example - that explicitly specifies this to be non-deterministic vs forEachOrdered), they do preserve encounter order. And your Stream.of does return an ordered stream; order that is not broken anywhere (via unordered for example, or sorted/distinct)

Eugene
  • 117,005
  • 15
  • 201
  • 306
  • 2
    Do you have a source for your assertion that terminal operations respect encounter order unless otherwise specified? – shmosel Jan 10 '18 at 20:46
  • @shmosel I remember reading this topic and it was Stuart Marks if I am not mistaken (or Holger?) saying this. Will update when I find it – Eugene Jan 10 '18 at 21:05
  • 2
    @shmosel there is no explicit statement about this policy, which isn’t a good situation, indeed, however, it’s a just fact that maintaining the order is not explicitly stated, i.e. even simple operations like `filter` or `map` don’t have a statement about maintaining the order. Even for `reduce` and `collect` you can only derive the maintenance of the order from the fact that the functions must be associative but are not required to be commutative. – Holger Jan 11 '18 at 12:58
  • @shmosel See the answer I just posted. TLDR: spec bug. I don't think I've discussed this particular anywhere else, but it's possible that I've forgotten. – Stuart Marks Jan 11 '18 at 23:52
  • @Holger See my comment above. – Stuart Marks Jan 11 '18 at 23:52
0

Given the fact the Stream.of returns, an ordered stream as stated in the documentation and none of the intermediate operations you've shown change this behavior we can, therefore, say that the returned iterator is guaranteed to provide the values 2, 3, 4 in order when enumerated because calling the iterator terminal operation on an ordered stream or sequence (e.g. a List implementation) should produce elements in that order when enumerated over.

So it shouldn't matter whether the code is executed sequentially or in parallel as long as we have an ordered source, intermediate operation(s) & terminal operation which respects this order then the order should be maintained.

Ousmane D.
  • 54,915
  • 8
  • 91
  • 126
  • What's your basis for saying `iterator()` respects the encounter order? – shmosel Jan 11 '18 at 00:22
  • @shmosel not saying this is for every case. but in regards to the provided example, it should respect the encounter order because the stream remains ordered. – Ousmane D. Jan 11 '18 at 00:29
  • That's a tautology. Just because a stream is ordered doesn't mean every operation will respect its order. `forEach()` doesn't. Many collectors don't. – shmosel Jan 11 '18 at 00:30
  • @shmosel Right, that's `forEach`... but with `Iterator` if a stream or a source has a specified ordering (preserves order) then `Iterator` should retrieve the elements in that order when enumerated. – Ousmane D. Jan 11 '18 at 00:36
  • @shmosel also note that as mentioned in my answer if the source & intermediate operations as well as the terminal operation respects the encounter order then it shouldn’t matter whether the stream is executed in parallel or not as we should maintain that encounter order. So I didn’t just say “just because a stream is ordered”. As for iterator the order in which elements are retrieved when enumerated are dependent on what type of collection or stream we are dealing with. – Ousmane D. Jan 11 '18 at 00:47
  • In the cases where the stream remains ordered then as mentioned Iterator(in this specific case) should retrieve those elements in order when enumerated. – Ousmane D. Jan 11 '18 at 00:49
  • Again, what's your source? – shmosel Jan 11 '18 at 00:50