Stream in production code

Question

Do people really use Scala's Stream class in production code, or is it primarily of academic interest?

What is so special about `Stream` that makes you ask this question? Why shouldn't one use it in production code? — Madoc, Mar 19 '11 at 16:07
It's easy to exhaust memory for one. Also I seem to recall reading that its implementation is showing its age. So I'm just curious whether people really use it for things beyond the Euler Project. ;) — Eric Bowman - abstracto -, Mar 19 '11 at 16:10

score 4 · Accepted Answer · answered Mar 19 '11 at 23:37

4

There's no problem with Stream, except when people use it to replace Iterator -- as opposed to replacing List, which is the collection most similar to it. In that particular case, one has to be careful in its use. On the other hand, one has to be careful using Iterator as well, since each element can only be iterated through once.

So, since both have their own problems, why single out Stream's? I daresay it's simply that people are used to Iterator from Java, whereas Stream is a functional thing.

answered Mar 19 '11 at 23:37

Daniel C. Sobral

295,120
86
501
681

Well, Stream does seem to have its problems, such as when used in reduceLeft. – Eric Bowman - abstracto - Mar 20 '11 at 08:46
A lot of good answers to this question; this one was for me the most helpful, since I didn't quite grok that using it like a List, not an Iterator, is the right approach. Thanks to everybody who replied. – Eric Bowman - abstracto - Mar 25 '11 at 13:03

score 3 · Answer 2 · edited May 23 '17 at 10:33

3

Even though I wrote that Iterator is what I want to use nearly all the time I do use Stream in production code. I just don't automatically assume that the cells are garbage collected.

Sometimes Stream fits the problem perfectly. I think the api gives some good examples where recursion is involved...

edited May 23 '17 at 10:33

Community

1
1

answered Mar 19 '11 at 20:12

huynhjl

41,520
14
105
158

Thanks for the link to the other question, very helpful. – Eric Bowman - abstracto - Mar 19 '11 at 20:32

score 3 · Answer 3 · answered Mar 20 '11 at 10:57

Look here. This blog post describes how to use Scala Streams (along with memory mapped file) to read large files (1-2G) efficiently.

I did not try it yet but the solution looks reasonable. Stream provides a nice abstraction on top of the low level ByteBuffer Java API to handle a memory mapped file as a sequence of records.

score 2 · Answer 4 · answered Mar 19 '11 at 17:54

2

Yes, I use it, although it tends to be for something like this:

(as.toStream collect expensiveConversionToB) match {
  case b #:: _ => //found my expensive b
  case _       =>
}

Of course, I might use a non-strict view and a find for this example

answered Mar 19 '11 at 17:54

oxbow_lakes

133,303
56
317
449

I guess what motivates me is the realization that Stream is so fragile in terms of memory use, that I would shudder to find it in production code, unless there was no other way. – Eric Bowman - abstracto - Mar 19 '11 at 20:34

score 1 · Answer 5 · answered Mar 19 '11 at 18:48

Since the only reason not to use Streams is that it can be tricky to ensure the JVM isn't keeping references to early conses around, one approach I've used that's fairly nice is to build up a Stream and immediately convert it to an Iterator for actual use. It loses a little of Stream's nice properties on the use side especially with respect to backtracking, but if you're only going to make one pass over the result it's frequently easier to build a structure this way than to contort into the hasNext/next() model of Iterator directly.

Stream in production code

5 Answers5