0

I have very large stream of objects. I read them from DB (in streaming mode of course) and return to user via REST in streaming mode (to avoid memory consumption).

I'd like to have them grouped (10 in group i.e.) during processing but still be represented by stream.

Something like:

Stream<Obj> -> Stream<List<Obj>> -> REST response

I dont want to group them with Collectors producing Map or List, I still want them as a stream to lower memory usage.

Are the Collectors my choice or what should I try? The best is to stay on the edge of Java 8 stream feature.

Tunaki
  • 132,869
  • 46
  • 340
  • 423
odiszapc
  • 4,089
  • 2
  • 27
  • 42
  • There is really nothing built-in (refer to this answer http://stackoverflow.com/a/34159174/1743880 and this answer http://stackoverflow.com/a/20507988/1743880). You will have to have an intermediate container like Map or List. – Tunaki Feb 25 '16 at 13:38
  • Not really clear what you are asking. Do want to split the stream into a stream of streams, sort of like [this (for Python)](http://stackoverflow.com/q/24527006/1639625)? – tobias_k Feb 25 '16 at 13:48
  • @tobias_k I want to split Stream of POJOs to Stream>. I work with simple Java Plain Objects. – odiszapc Feb 25 '16 at 13:53
  • @odiszapc That much is clear, but how? What will those lists contain? Will you split a stream of 100 elements into a stream of 10 lists with ten elements each? – tobias_k Feb 25 '16 at 13:59
  • @tobias_k Exactly. Iterating them from left to right. First 10 records, second 10 records and so on. It's easy using iteratore, I want to do this using stream API – odiszapc Feb 25 '16 at 14:02
  • @tobias_k I need lazy impl, guys in that topic said it's not possible for lazy stream to do partition. My stream has no random access, unfortunately. Looks like task could not be solved using Stream API – odiszapc Feb 25 '16 at 14:04
  • Have you tried turning the stream into an iterator then? `stream.iterator()` Or does that complain about the stream being closed after the first chunk? – tobias_k Feb 25 '16 at 14:05
  • Basically my DB returns iterator itself, and I convert it to stream. So, iterator is the thing I already have. – odiszapc Feb 25 '16 at 14:07
  • 2
    So, why don't you just stick with Iterators and use [Guava's `Iterators.partition`](http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Iterators.html#partition%28java.util.Iterator,%20int%29) instead? – tobias_k Feb 25 '16 at 14:31
  • @tobias_k OMG, you save my century ^) – odiszapc Feb 26 '16 at 07:41

0 Answers0