Are Options and named default arguments like oil and water in a Scala API?

Question

I'm working on a Scala API (for Twilio, by the way) where operations have a pretty large amount of parameters and many of these have sensible default values. To reduce typing and increase usability, I've decided to use case classes with named and default arguments. For instance for the TwiML Gather verb:

case class Gather(finishOnKey: Char = '#', 
                  numDigits: Int = Integer.MAX_VALUE, // Infinite
                  callbackUrl: Option[String] = None, 
                  timeout: Int = 5
                  ) extends Verb

The parameter of interest here is callbackUrl. It is the only parameter which is really optional in the sense that if no value is supplied, no value will be applied (which is perfectly legal).

I've declared it as an option in order to do the monadic map routine with it on the implementation side of the API, but this puts some extra burden on the API user:

Gather(numDigits = 4, callbackUrl = Some("http://xxx"))
// Should have been
Gather(numDigits = 4, callbackUrl = "http://xxx")

// Without the optional url, both cases are similar
Gather(numDigits = 4)

As far as I can make out, there are two options (no pun intended) to resolve this. Either make the API client import an implicit conversion into scope:

implicit def string2Option(s: String) : Option[String] = Some(s)

Or I can redeclare the case class with a null default and convert it to an option on the implementation side:

case class Gather(finishOnKey: Char = '#', 
                  numDigits: Int = Integer.MAX_VALUE, 
                  callbackUrl: String = null, 
                  timeout: Int = 5
                  ) extends Verb

My questions are as follows:

1. Are there any more elegant ways to solve my particular case?
2. More generally: Named arguments is a new language feature (2.8). Could it turn out that Options and named default arguments are like oil and water? :)
3. Might using a null default value be the best choice in this case?

Answer 1

Here's another solution, partly inspired by Chris' answer. It also involves a wrapper, but the wrapper is transparent, you only have to define it once, and the user of the API doesn't need to import any conversions:

class Opt[T] private (val option: Option[T])
object Opt {
   implicit def any2opt[T](t: T): Opt[T] = new Opt(Option(t)) // NOT Some(t)
   implicit def option2opt[T](o: Option[T]): Opt[T] = new Opt(o)
   implicit def opt2option[T](o: Opt[T]): Option[T] = o.option
}

case class Gather(finishOnKey: Char = '#', 
                  numDigits: Opt[Int] = None, // Infinite
                  callbackUrl: Opt[String] = None, 
                  timeout: Int = 5
                 ) extends Verb

// this works with no import
Gather(numDigits = 4, callbackUrl = "http://xxx")
// this works too
Gather(numDigits = 4, callbackUrl = Some("http://xxx"))
// you can even safely pass the return value of an unsafe Java method
Gather(callbackUrl = maybeNullString())

---

To address the larger design issue, I don't think that the interaction between Options and named default parameters is as much oil-and-water as it might seem at first glance. There's a definite distinction between an optional field and one with a default value. An optional field (i.e. one of type Option[T]) might never have a value. A field with a default value, on the other hand, simply does not require its value to be supplied as an argument to the constructor. These two notions are thus orthogonal, and it's no surprise that a field may be optional and have a default value.

That said, I think a reasonable argument can be made for using Opt rather than Option for such fields, beyond just saving the client some typing. Doing so makes the API more flexible, in the sense that you can replace a T argument with an Opt[T] argument (or vice-versa) without breaking callers of the constructor[1].

As for using a null default value for a public field, I think this is a bad idea. "You" may know that you expect a null, but clients that access the field may not. Even if the field is private, using a null is asking for trouble down the road when other developers have to maintain your code. All the usual arguments about null values come into play here -- I don't think this use case is any special exception.

[1] Provided that you remove the option2opt conversion so that callers must pass a T whenever an Opt[T] is required.

Answer 2

Don't auto-convert anything to an Option. Using my answer here, I think you can do this nicely but in a typesafe way.

sealed trait NumDigits { /* behaviour interface */ }
sealed trait FallbackUrl { /* behaviour interface */ }
case object NoNumDigits extends NumDigits { /* behaviour impl */ }
case object NofallbackUrl extends FallbackUrl { /* behaviour impl */ }

implicit def int2numd(i : Int) = new NumDigits { /* behaviour impl */ }
implicit def str2fallback(s : String) = new FallbackUrl { /* behaviour impl */ }

class Gather(finishOnKey: Char = '#', 
              numDigits: NumDigits = NoNumDigits, // Infinite
              fallbackUrl: FallbackUrl = NoFallbackUrl, 
              timeout: Int = 5

Then you can call it as you wanted to - obviously adding your behaviour methods to FallbackUrl and NumDigits as appropriate. The main negative here is that it is a ton of boilerplate

Gather(numDigits = 4, fallbackUrl = "http://wibble.org")

Answer 3

Personally, I think using 'null' as default value is perfectly OK here. Using Option instead of null is when you want to convey to your clients that something may not be defined. So a return value may be declared Option[...], or a method arguments for abstract methods. This saves the client from reading documentation or, more likely, get NPEs because of not realizing something may be null.

In your case, you are aware that a null may be there. And if you like Option's methods just do val optionalFallbackUrl = Option(fallbackUrl) at the start of the method.

However, this approach only works for types of AnyRef. If you want to use the same technique for any kind of argument (without resulting to Integer.MAX_VALUE as replacement for null), then I guess you should go with one of the other answers

Answer 4

I think as long as no language support in Scala for a real kind of void (explanation below) ‘type’, using Option is probably the cleaner solution in the long run. Maybe even for all default parameters.

The problem is, that people who use your API know that some of your arguments are defaulted might as well handle them as optional. So, they’re declaring them as

var url: Option[String] = None

It’s all nice and clean and they can just wait and see if they ever get any information to fill this Option.

When finally calling your API with a defaulted argument, they’ll face a problem.

// Your API
case class Gather(url: String) { def this() = { ... } ... }

// Their code
val gather = url match {
  case Some(u) => Gather(u)
  case _ => Gather()
}

I think it would be much easier then to do this

val gather = Gather(url.openOrVoid)

where the *openOrVoid would just be left out in case of None. But this is not possible.

So you really should think about who is going to use your API and how they are likely to use it. It may well be that your users already use Option to store all variables for the very reason that they know they are optional in the end…

Defaulted parameters are nice but they also complicate things; especially when there is already an Option type around. I think there is some truth in your second question.

Answer 5

Might I just argue in favor of your existing approach, Some("callbackUrl")? It's all of 6 more characters for the API user to type, shows them that the parameter is optional, and presumably makes the implementation easier for you.

Answer 6

I think you should bite the bullet and go ahead with Option. I have faced this problem before, and it usually went away after some refactoring. Sometimes it didn't, and I lived with it. But the fact is that a default parameter is not an "optional" parameter -- it's just one that has a default value.

I'm pretty much in favor of Debilski's answer.

Answer 7

I was also surprised by this. Why not generalize to:

implicit def any2Option[T](x: T): Option[T] = Some(x)

Any reason why that couldn't just be part of Predef?