14

As I understand, Stream retains the recently evaluated elements. I guess it does not retain all evaluated elements (it is not feasible), so it probably uses some internal "cache".

Is it correct? Can I control the size and policies of this cache?

Michael
  • 10,185
  • 12
  • 59
  • 110
  • 1
    You shouldn't think of `Stream`s as `Iterator`s, but rather as `List`s whose evaluation can be delayed. [This question and its answers](http://stackoverflow.com/questions/1527962/difference-between-iterator-and-stream-in-scala) may help. – Philippe Dec 19 '11 at 20:40

2 Answers2

14

Streams are like lists that generate their members as they are required. Once an element has been generated, it is retained in the stream and reused.

For example:

lazy val naturals: Stream[Int] = Stream.cons(0, naturals.map{_ + 1})

will give you a stream of the natural numbers. If I call

naturals(5)

it will generate elements 0-5 and return 5, if I then call

naturals(8)

It will reuse the first 6 elements and generate 3 more.

If you are concerned about memory usage, you can use Stream.drop(num) to produce a new stream with num elements removed from the beginning, allowing the truncated elements to be garbage collected with the old stream. For example:

naturals(5) //returns 5
val truncated = naturals.drop(4)
truncated(5) //returns 9
Dan Simon
  • 12,891
  • 3
  • 49
  • 55
  • 7
    However, as long as `naturals` is still pointing to the head of the stream, those first five elements will not be garbage collected. – Daniel C. Sobral Dec 20 '11 at 00:10
8

The Stream-object retains all references that have been evaluated/accessed so far. Stream works like a List. Every element that can be reached from a held reference, and which has already been accessed at least once, won't be garbage collected.

So basically your pointers into the stream and what you have evaluated so far define what will get cached.

ziggystar
  • 28,410
  • 9
  • 72
  • 124
  • So, `stream` does not have a _limited_ memory cache to store its elements. Sounds like a very memory-consuming approach ... – Michael Dec 19 '11 at 19:56
  • 2
    This is why you should _never_ hold a reference to any streams. Always define them using `def`. See also http://stackoverflow.com/questions/12486762/scala-tail-recursive-stream-processor-function-defined-in-trait-holds-reference and http://stackoverflow.com/questions/12529697/how-to-write-non-leaking-tail-recursive-function-using-stream-cons-in-scala – ron Sep 21 '12 at 11:40