6

What is the difference and the term name between these underscore usage from these codes: (see the handler(resource) part)

1.

def readFile[T](f: File)(handler: FileInputStream => Byte => T): T = {
    val resource = new java.io.FileInputStream(f)
    try {
        val hh = handler(resource)_
        hh(2)
    } finally {
        resource.close()
    }
}

val bs = new Array[Byte](4)

readFile(new File("scala.txt")) {
    input => b: Byte => println("Read: " + (input.read(bs) + b))
}

I got compile error:

Error:(55, 29) _ must follow method; cannot follow Byte => T
            val hh = handler(resource)_
                        ^

What does it mean?

2.

def readFile[T](f: File)(handler: FileInputStream => Byte => T): T = {
    val resource = new java.io.FileInputStream(f)
    try {
        val hh = handler(resource) _
        hh(2)
    } finally {
        resource.close()
    }
}

// Lower parts are same, so removed for brevity...
// ...

The result is same as no. 1, I got: _ must follow method compile error.

I read this is because the underscore is used to convert method to function (ETA Expansion), but I also seen the same underscore is used for Partial Applied Function without problem, for example:

val sum = (x: Int, y: Int) => x + y
val sum2 = sum _

No error in this case.

3.

def readFile[T](f: File)(handler: FileInputStream => Byte => T): T = {
    val resource = new java.io.FileInputStream(f)
    try {
        val hh = handler(resource)(_)
        hh(2)
    } finally {
        resource.close()
    }
}

//...

This one works fine. If I'm not wrong, the underscore in this case is called ETA Expansion, is that correct? But I also read from this Q/A, this kind of underscore is for Partial Applied Function. In same page, someone also said this is a Placeholder syntax. So which one is the correct one?

4.

def readFile[T](f: File)(handler: FileInputStream => Byte => T): T = {
    val resource = new java.io.FileInputStream(f)
    try {
        val hh = handler(resource)
        hh(2)
    } finally {
        resource.close()
    }
}

//...

This one doesn't use underscore but it works fine too just like no. 3. My question for this case, what is the difference with no. 3? Should I use no. 4 than no. 3? Is the underscore redundant in no. 3?

Sorry for a lot of questions here, but that underscore thing is really confusing.

Somehow I thought the complexity of underscore in Scala matches the complexity of pointer and reference (*/&/&&) in C/C++.

UPDATE:

5.

I found something interesting again about the underscore:

scala> def sum(x: Int, y: Int) = x + y     // sum is a method because of def
sum: (x: Int, y: Int)Int

scala> val sum2 = sum _    // sum2 is explicit ETA expanded function from sum
sum2: (Int, Int) => Int = <function2>

scala> sum2(2,3)      // testing
res0: Int = 5

scala> val sum3 = (x: Int, y: Int) => x + y      // sum3 is a function object
sum3: (Int, Int) => Int = <function2>

scala> val sum4 = sum3 _           // what happpened here?
sum4: () => (Int, Int) => Int = <function0>

scala> sum4()(2,3)
res2: Int = 5

Could you tell me what happened to sum4? Why the result of sum3 _ has function type: () => (Int, Int) => Int?

6.

List(1, 2, 3) foreach println _

According to this answer, this is Partially applied functions. Ok, I can see that the space before underscore is kinda tricky. It is actually same as:

List(1, 2, 3).foreach(println(_))

So this is indeed Partially applied function.

But if I did this:

scala> List(1, 2, 3).foreach(println _+1)  //#1
<console>:8: error: type mismatch;
 found   : Int(1)
 required: String
              List(1, 2, 3).foreach(println _+1)
                                          ^

scala> List(1, 2, 3).foreach(println _+"#")    //#2 printed out nothing (why?)

scala> List(1, 2, 3).foreach(println 1+_)      //#3
<console>:1: error: ')' expected but integer literal found.
       List(1, 2, 3).foreach(println 1+_)
                                     ^

scala> List(1, 2, 3).foreach(println "#"+_)    //#4
<console>:1: error: ')' expected but string literal found.
       List(1, 2, 3).foreach(println "#"+_)
                                     ^

Newcomer usually will think the underscore in this case as placeholder, but I believe it isn't, isn't it?

Community
  • 1
  • 1
null
  • 8,669
  • 16
  • 68
  • 98

1 Answers1

7

1 & 2 - are same and this is eta-expansion, which means that function is getting converted from just function as part of language to the real object of some FunctionN class:

scala> def f(a: Int) = a
f: (a: Int)Int

scala> f.apply(1)
<console>:9: error: missing arguments for method f;
follow this method with `_' if you want to treat it as a partially applied function
              f.apply(1)
              ^
scala> f _
res1: Int => Int = <function1>    

scala> (f _).apply(1)
res2: Int = 1

It doesn't work in your example as handler(resource) is an expression which returns function-object Byte => T (as handler is a function-object FileInputStream => Byte => T and you did partial applying on it), so scala can't do eta-expansion for expressions (only for values and methods).

4 is partially applied as side effect of scala's curried functions support (by curried i mean ability to take parameters one-by-one).

3 is just explicitly partially applied.

Note that in all 3 examples - your handler: FileInputStream => Byte => T function is an object (so it's already eta-expanded), if you try to do same things with multi-parameter-list methods (which is not yet expanded to the curried function) - you will receive the opposite results for 1&2&4:

scala> def f(a: Int)(b: Int) = a //it's not a curried function, as it's just multi-parameter-list method
f: (a: Int)(b: Int)Int

scala> f(2) 
<console>:9: error: missing arguments for method f;
follow this method with `_' if you want to treat it as a partially applied function
              f(2)
           ^
scala> f(2) _ //you need to convert f(2) to object first
res4: Int => Int = <function1>

scala> f(2)(_)
res5: Int => Int = <function1>

scala> f _  //expand method to the function object
res6: Int => (Int => Int) = <function1>

So partial application also do an eta-expansion for you if needed. You may also think about eta-expansion as (not precisely) function with 0 partially-applied arguments, so it's pretty much close terms, as partially-applied functions are always objects in scala (in haskell it's first-class function) because you always need partially applied function to be first-class-citizen (like object or f-c-function) to apply it after eta-expansion.

5. Scala can do eta-expansion for values itself, as they may be considered as compile-time functions with 0 parameters (that's why you see () => ...). It can expand any value to the function object:

scala> val k = 5
k: Int = 5

scala> val kk = k _
kk: () => Int = <function0>

scala> val kkk = kk _
kkk: () => () => Int = <function0>

scala> 

In your example - value is just another function-object itself. Also (Int, Int) => Int is not fully curried function (it's taking parameters some-count by some-count), but scala can also do automatical partial applying for such. To make it fully curried:

scala> def f(a: Int, b: Int) = a
f: (a: Int, b: Int)Int

scala> (f _).curried
res23: Int => (Int => Int) = <function1>

scala> def f(a: Int, b: Int)(z: Int) = a
f: (a: Int, b: Int)(z: Int)Int

scala> (f _).curried
res22: Int => (Int => (Int => Int)) = <function1>

This process actually called currying.

Another way to make it curried - is using tuples. It's not so pure as currying is actually removing tuples , but scala's Tuple is just a class and not tuple in parameter list: (Int, Int) => Int - input is not a tuple in scala's terminology, but in ((Int, Int)) => Int, input is a tuple (regardless that from FP-perspecive it's a tuple of to objects in first case and tuple of one object in second). Example of pseudo-tuplying:

 scala> def f(a: Int, b: Int) = a
 f: (a: Int, b: Int)Int

 scala> (f _).tupled
 res24: ((Int, Int)) => Int = <function1>

5 vs 1&2 As you've seen before you can't apply eta-expansion to the expression, only methods/values/vars:

 scala> 5 _
 <console>:8: error: _ must follow method; cannot follow Int(5)
          5 _
          ^

 scala> val (a, b) = (5, 5)

 scala> (a + b) _
 <console>:10: error: _ must follow method; cannot follow Int
              (a + b) _
                 ^

You see the "method" requiring in the error message, but scala aims to treat methods/values/vars (when they are members of class/object) in the same way to (at least partially) support UAP.

6 It's eta expansion which returns Function0 by default:

scala> val a = println _
a: () => Unit = <function0>

You may expect function1 here, but println is overloaded and eta-expansion mechanism choose the fewest signature. When other type is expected (like in Function1 in foreach) - it may choose another:

scala> val a: String => Unit = println _
a: String => Unit = <function1>

As i said you may consider function-object as function partially applied with 0 arguments (which includes eta-expansion if needed), so that's the source of confusion with another answer (i would choose better example there).

As i said in P.S.2 this expansion may be applied automatically:

scala> List(1,2) foreach println
1
2

About println _ +"#" - it works because any class (including Function1) in scala has implicit def + (s: String) (that's why Int doesn't work here) defined in Predef (see SI-194):

scala> println _
res50: () => Unit = <function0>

scala> println _ + "#"
res51: String = <function0>#

Every other options doesn't work because of 5 vs 1&2, actually scala can't even parse string after one-parameter function:

scala> println "#"
<console>:1: error: ';' expected but string literal found.
   println "#"
           ^

You should specify object host to fix it as scala expects something like "obj method param" (but this is experimental feature and sometimes you need to paste some empty lines or ";" to make it work):

scala> Predef println "aaa"
aaa

P.S. About C++ reference/pointer. Function has no value as it's compile-time structure, so compiler is just creating a value for it, that process is called eta-expansion (or eta-abstraction for pure functional). This value may be part of pointer (an object with reference to it) or just reference itself - doesn't matter. What is matter is that function moves from compile (method) to the runtime (f-c-function) here, so it "becomes alive".

P.S.2. Sometimes scala do eta-expansion automatically (like here) when partially applied multi-parameter-list method is explicitly passed as parameter.

P.S.N. You may find some additional info about underscore in @Daniel C. Sobral answer about scala punctuation.

Community
  • 1
  • 1
dk14
  • 22,206
  • 4
  • 51
  • 88
  • Thanks a lot for the answer. But I don't understand with your saying: `Note that in all 3 examples - your function is an object (so it's already eta-expanded), if you will try to do same things with multi-parameter methods - you will receive opposite results for 1&2&4`. Could you elaborate more? Which function do you mean? The `readFile` or the `handler(resource)` ? – null Jan 01 '15 at 17:23
  • 1
    i mean `handler: FileInputStream => Byte => T` is an object (so it's already eta-expanded) – dk14 Jan 01 '15 at 17:25
  • 1
    for example, if you have some method like `f(a: FileInputStream)(b: Byte)T` - you have to expand it to the function-object first before passing to the `readFile`: `readFile(file)(f _)`, but actually scala can do it (eta expansion) automatically – dk14 Jan 01 '15 at 17:49
  • Great answer! But I'm still confused when you said `//there is no ability for currying here, as it's just multi-parameter-list method`. What does actually `currying` mean here? Then you also said `It doesn't work in your example as handler(resource) is already expanded to Byte => T (see currying) `. Did you mean the `readFile` is already a `curried function` instead? If I'm not wrong, from what I read, currying is the process of conversion (for example) from `(Int, Int) => Int` to `Int => Int => Int`, the resulting function type is what known as curried function. – null Jan 01 '15 at 20:06
  • I have added case no. 5 in the question, could you elaborate it what happened there? – null Jan 01 '15 at 20:33
  • 1
    `readFile` is not a function - it's a method, it can become a function only if you do eta-expansion for. But you're right that by currying i actually meant curried function, so corrected my answer a bit. – dk14 Jan 02 '15 at 02:39
  • So in case no. 5, the process of `sum3` to `sum4` is actaully an ETA expansion too? If you don't mind, could you look at case 6? :) – null Jan 02 '15 at 08:16
  • 1
    and yes, because `sum4` interpreted as just value (not function) there, so simply saying `() =>` is added before it by `_` – dk14 Jan 02 '15 at 09:09
  • What's the difference between `List(1,2,3).foreach(println _)` and ` List(1,2,3).foreach(println _+"#")` ? The former printed out `1 2 3` and latter printed out nothing. What I understand for the former is: with ETA expansion, it converted `println` method before the `_` to function type of `foreach` expected which is `Function1`, thus it executed the `println(x: Any)`. Now for the latter, what happened? Did the ETA conversion failed because of `+"#"`? You said `it works because any class (including Function1) in scala has implicit def + (s: String)`, but I want to know why it printed nothing? – null Jan 02 '15 at 11:41
  • 1
    just think a little...`println _` is an object of class `Function1` (with `toString = ""`) you did `+ "#"` to this object and received string "#", which was returned to `foreach` - `foreach` ignored result, so it did nothing. println didn't executed as you didn't applied it to anything - you just turned it to the string. try for example to call `anyobjectyouwant + "#"`, or just call separately `println _ + "#"` - and you will see. – dk14 Jan 02 '15 at 11:52
  • p.s. by Function0, Function1 etc i mean real normal scala's classess http://www.scala-lang.org/api/current/index.html#scala.Function1 – dk14 Jan 02 '15 at 11:56
  • I see, I get it now :). So the `println _ + 1` failed because `println _` turned into function object (Function0), but the `Function0` doesn't have `def +(x: Int)`? One more question again, the `println "#"` resulted in error, is it because scala method's parameter should be enclosed with parenthesis i.e. `println("#")` ? Or is there another reason? – null Jan 02 '15 at 12:11
  • 1
    yes :) , that's the reason, the only exception is infix notation for methods with one or zero parameters (if object-host of method is explicitly present), like `a + b` or `a anymethod b` or even sometimes `a anymethodofa` (experimental), scala could parse it, because such sequence will be identified by first expression's (not method) token ("a"). – dk14 Jan 02 '15 at 12:55
  • 1
    `Predef println "aaa"` will print you "aaa" (but it's experimental feature) – dk14 Jan 02 '15 at 12:56
  • Hi, please see [this answer](http://stackoverflow.com/a/17484678/844005): why the ETA expansion doesn't work for local val variable? – null Jan 02 '15 at 15:35
  • 1
    as I said, but forgot to mentaion that it's only for class/object members, added to my answer, thanks. It doesn't work as UAP is formulated only for OOP, so it's about members only (but honestly scala could do it for locals). – dk14 Jan 02 '15 at 17:03
  • 1
    by the way, scala's UAP isn't full - see http://stackoverflow.com/a/27624879/1809978 – dk14 Jan 02 '15 at 17:09
  • Thank you very much. You are being helpful all this time. I wish I can upvote your answer multiple times. Maybe this is the final question ^^, I have a small question related with underscore so I put in [pastebin](http://pastebin.com/w01t1Gkh). What do you think? Is it possible to achieve that? – null Jan 02 '15 at 17:56
  • 1
    You're welcome. It's sad to say but this seems to be impossible, as underscore here is just syntax sugar for lambda and it's restricted by braces, in other words lambda always will be induced inside braces `println("a" + _)` === `println(x => "a" + x)`, when you need `x => println("a" + x)` the only way i see is using functors `List(1,2,3).map(_ + "a").foreach(println)`. Actually you could upvote my answer about incomplete UAP, I mentioned :). – dk14 Jan 02 '15 at 18:23
  • Upvoted :). Btw, what is the meaning of functor? What functional programming language would you recommend besides scala? – null Jan 02 '15 at 18:52
  • 1
    Thanks! Haskell is very good for that purpose, it's better than scala as functional language, but Scala has Java and OOP compatibility, so it's better solution for enterprise as result. Simply saying, Functor is any type which has defined `map` (Lists are functors for example) on it, Monad should have flatMap (or map + flatten or map + concat + fold). Not simply saying Functor - is a morphism between categories: http://en.wikipedia.org/wiki/Functor – dk14 Jan 03 '15 at 06:06
  • you could also read http://fef.ogu.edu.tr/matbil/eilgaz/kategori.pdf and this question (and future answers) http://stackoverflow.com/questions/27752424/what-is-the-analog-of-category-in-programming/27752428 to better understand category theory. – dk14 Jan 03 '15 at 07:04
  • I know you would say Haskell :). What is the example of Monad in scala? It seems Monad is a complex subject, but still, could you explain what is Monad in simple words? – null Jan 03 '15 at 07:58
  • it's also a `List` as it has `flatMap` operation defined on it :)) You could also look at Lisp/Scheme but it's much different from scala. – dk14 Jan 03 '15 at 08:47