Scala lazy elements in iterator

Question

Does anyone know how to create a lazy iterator in scala?

For example, I want to iterate through instantiating each element. After passing, I want the instance to die / be removed from memory.

If I declare an iterator like so:

val xs = Iterator(
 (0 to 10000).toArray,
 (0 to 10).toArray,
 (0 to 10000000000).toArray)

It creates the arrays when xs is declared. This can be proven like so:

def f(name: String) = {
  val x =  (0 to 10000).toArray
  println("f: " + name) 
  x
}

val xs = Iterator(f("1"),f("2"),f("3"))

which prints:

scala> val xs = Iterator(f("1"),f("2"),f("3"))
f: 1
f: 2
f: 3
xs: Iterator[Array[Int]] = non-empty iterator

Anyone have any ideas?

Streams are not suitable because elements remain in memory.

Note: I am using an Array as an example, but I want it to work with any type.

This hack seems to perform as I want: val it = List(() => g("1"), () => g("2"), () => g("3")).toIterator.map(_()) — S0rin, Jan 18 '13 at 11:08

score 2 · Answer 1 · edited May 23 '17 at 11:46

2

Scala collections have a view method which produces a lazy equivalent of the collection. So instead of (0 to 10000).toArray, use (0 to 10000).view. This way, there will be no array created in the memory. See also https://stackoverflow.com/a/6996166/90874, https://stackoverflow.com/a/4799832/90874, https://stackoverflow.com/a/4511365/90874 etc.

edited May 23 '17 at 11:46

Community

1
1

answered Jan 18 '13 at 10:48

thSoft

21,755
5
88
103

If I had them in a list, they would remain in memory for the duration of the list. I only want them in memory when I pass over that element. – S0rin Jan 18 '13 at 10:52
I am using an Array as an example, but I want it to work with any type. – S0rin Jan 18 '13 at 11:08
@SomeoneElse: Having been placed in a List does not imply being ineligible for reclamation by the garbage collector. If you build a list of 10 items and `drop` the first 5, e.g., then those 5 cons cells will be garbage, as will the content values (assuming nothing else retains references to those list cells or their contained values, of course). – Randall Schulz Jan 18 '13 at 14:52

score 1 · Answer 2 · answered Jan 18 '13 at 11:15

Use one of Iterator factory methods which accepts call-by-name parameter.

For your first example you can do one of this:

val xs1 = Iterator.fill(3)((0 to 10000).toArray)
val xs2 = Iterator.tabulate(3)(_ => (0 to 10000).toArray)
val xs3 = Iterator.continually((0 to 10000).toArray).take(3)

Arrays won't be allocated until you need them.

In case you need different expressions for each element, you can create separate iterators and concatenate them:

val iter = Iterator.fill(1)(f("1")) ++ 
           Iterator.fill(1)(f("2")) ++ 
           Iterator.fill(1)(f("3"))

Scala lazy elements in iterator

2 Answers2