if a function is able to be optimized for tail recursion then why isn't this optimization automatically applied
It is.
Unfortunately, I haven't found a quote from the SLS which would guarantee this, though.
why do I need to mark a function that can be optimized with @tailrec
?
Note: Scala doesn't guarantee proper tail recursion for functions, only for methods!
You don't annotate methods that can be optimized. You annotate methods that must be optimized, so that you will get a compile error, when they can't be optimized.
See the documentation for scala.annotation.tailrec
:
A method annotation which verifies that the method will be compiled with tail call optimization.
If it is present, the compiler will issue an error if the method cannot be optimized into a loop.
The documentation is misleading in exactly what is optimized ("tail call optimization", when really Scala only optimizes direct tail recursion) but it is clear about the purpose of the annotation.
The reason for this annotation is that sometimes people's intuition about what is and isn't direct tail recursion may be wrong. There are plenty of questions here on SO of the form "why doesn't Scala optimize my tail-recursive method" whose answer is "because it isn't tail-recursive". (Here is an example of a method where the fact that it can't be optimized is non-obvious.) So, by annotating a method, you signal both to the compiler and your fellow developers that this method must be optimized.