Confusing scala syntax

Question

trying to understand some scala syntax, and where to find their spec. Bellow i am confused about statefulMapConcat.

The signature is this one:

def statefulMapConcat[T](f: () => Out => immutable.Iterable[T]): Repr[T]

and we have

"be able to restart" in {
      Source(List(2, 1, 3, 4, 1))
        .statefulMapConcat(() => {
          var prev: Option[Int] = None
          x => {
            if (x % 3 == 0) throw ex
            prev match {
              case Some(e) =>
                prev = Some(x)
                (1 to e).map(_ => x)
              case None =>
                prev = Some(x)
                List.empty[Int]
            }
          }
        })
        .withAttributes(ActorAttributes.supervisionStrategy(Supervision.restartingDecider))
        .runWith(TestSink.probe[Int])
        .request(2)
        .expectNext(1, 1)
        .request(4)
        .expectNext(1, 1, 1, 1)
        .expectComplete()
    }

I do not under well how

{
          var prev: Option[Int] = None
          x => {
            if (x % 3 == 0) throw ex
            prev match {
              case Some(e) =>
                prev = Some(x)
                (1 to e).map(_ => x)
              case None =>
                prev = Some(x)
                List.empty[Int]
            }
          }

correspond to

Out => immutable.Iterable[T]

Can someone decompose it for me please ?

in my mind x would correspond to Out, but then we have the declaration of a variable before var prev: Option[Int] = None

I would like to understand and where to find explanation about those scala magic in general

https://stackoverflow.com/questions/13872196/multiline-function-literal-as-arguments-in-scala/13873899 — som-snytt, Dec 24 '19 at 00:34
That's my only useful SO answer. It says your thing in braces is a block, and the "result expression" at the end of the block is the function. — som-snytt, Dec 24 '19 at 00:35

score 5 · Answer 1 · answered Dec 24 '19 at 00:52

5

Two key concepts to understand here are

the value of the whole block expression is the value of the last expression in the block, and
functions are first-class values

Consider the following block expression

{
  var prev: Option[Int] = None
  42 // I am the last value of the block
}

Because the last expression in the block evaluates to value 42, the value of the whole block is 42 of type Int.

Similarly, consider the following block expression

{
  var prev: Option[Int] = None
  (x: Int) => { x + 1 } // I am also the last *value* of the block
}

Because the last expression in the block evaluates to value (x: Int) => { x + 1 }, the value of the whole block is value (x: Int) => { x + 1 } of type Int => Int.

In both cases, because expression var prev: Option[Int] = None is not the last expression in the block, it does not affect the type of the value of the whole block.

answered Dec 24 '19 at 00:52

Mario Galic

47,285
6
56
98

Thanks good illustration. Although in that specific context, what is the scope of var ? var prev: Option[Int] = None , just to be clear ..... it sounds like it is the variable that maintain the state.... So basically you declare a variable in the operator, and the operator keep calling the anonymous function. This sounds rather strange... Meaning the Operator StatefulMapcontact does not natively provide a way to maintain the state ... very very strange – MaatDeamon Dec 24 '19 at 14:10
1

To use precise words, a block has statements and a result expression. The compiler will warn about a "pure expression in statement position", when it notices. To answer your question about what happens to the var, which is no longer on the stack but heap allocated, I edited my answer. This is actually a performance gotcha and there ought to be a lint warning of some kind. Java doesn't allow it. – som-snytt Dec 25 '19 at 17:42
Another key concept is that the single arg to a function can be a block, as in my example. – som-snytt Dec 25 '19 at 17:52

som-snytt · Answer 2 · 2019-12-25T17:54:48.450

Quick illustration. Actually, no interior braces when you write the single arg as a block:

scala> def f(g: () => String => Int) = g()("42")
f: (g: () => String => Int)Int

scala> f(() => _.toInt)
res0: Int = 42

scala> f { () => s => s.toInt }
res1: Int = 42

scala> f ( () => s => s.toInt )
res2: Int = 42

scala> f { val x = "0" ; () => val y = x + "0" ; s => (s+y).toInt }
res3: Int = 4200

Showing with -Vprint:parser (-Xprint on 2.12) that it parses correctly, even though the semicolons might fool the eye:

    val res3 = f({
      val x = "0";
      (() => {
        val y = x.$plus("0");
        ((s) => s.$plus(y).toInt)
      })
    })

Edit: the magically captured var is on the heap:

scala> { var i = 42 ; (j: Int) => j + i }
res1: Int => Int = $$Lambda$892/0x00000001006ec840@6e981e78

scala> :javap -c -
Compiled from "<console>"
public class $line4.$read$$iw$$iw$ {
  public static final $line4.$read$$iw$$iw$ MODULE$;

  public static {};
    Code:
       0: new           #2                  // class $line4/$read$$iw$$iw$
       3: dup
       4: invokespecial #25                 // Method "<init>":()V
       7: putstatic     #27                 // Field MODULE$:L$line4/$read$$iw$$iw$;
      10: bipush        42
      12: invokestatic  #33                 // Method scala/runtime/IntRef.create:(I)Lscala/runtime/IntRef;
      15: astore_0
      16: aload_0
      17: invokedynamic #52,  0             // InvokeDynamic #0:apply$mcII$sp:(Lscala/runtime/IntRef;)Lscala/runtime/java8/JFunction1$mcII$sp;
      22: putstatic     #54                 // Field res1:Lscala/Function1;
      25: return

  public scala.Function1<java.lang.Object, java.lang.Object> res1();
    Code:
       0: getstatic     #54                 // Field res1:Lscala/Function1;
       3: areturn

  public static final int $anonfun$res1$1(scala.runtime.IntRef, int);
    Code:
       0: iload_1
       1: aload_0
       2: getfield      #65                 // Field scala/runtime/IntRef.elem:I
       5: iadd
       6: ireturn

  public $line4.$read$$iw$$iw$();
    Code:
       0: aload_0
       1: invokespecial #66                 // Method java/lang/Object."<init>":()V
       4: return
}

I think I made it look like Lady Gaga without her eyebrows in that movie. — som-snytt, Dec 24 '19 at 00:44

Confusing scala syntax

2 Answers2