3

This might be a really dumb question but I am trying to understand the logic behind using #flatMap and not just #map in this method definition in Finatra's HttpClient definition:

def executeJson[T: Manifest](request: Request, expectedStatus: Status = Status.Ok): Future[T] = {
  execute(request) flatMap { httpResponse =>
    if (httpResponse.status != expectedStatus) {
      Future.exception(new HttpClientException(httpResponse.status, httpResponse.contentString))
    } else {
      Future(parseMessageBody[T](httpResponse, mapper.reader[T]))
        .transformException { e =>
          new HttpClientException(httpResponse.status, s"${e.getClass.getName} - ${e.getMessage}")
        }
    }
  }
}

Why create a new Future when I can just use #map and instead have something like:

execute(request) map { httpResponse =>
  if (httpResponse.status != expectedStatus) {
    throw new HttpClientException(httpResponse.status, httpResponse.contentString)
  } else {
    try {
      FinatraObjectMapper.parseResponseBody[T](httpResponse, mapper.reader[T])
    } catch {
      case e => throw new HttpClientException(httpResponse.status, s"${e.getClass.getName} - ${e.getMessage}")
    }
  }
}

Would this be purely a stylistic difference and using Future.exception is just better style in this case, whereas throwing almost looks like a side-effect (in reality it's not, as it doesn't exit the context of a Future) or is there something more behind it, such as order of execution and such?

Tl;dr: What's the difference between throwing within a Future vs returning a Future.exception?

Mark Rotteveel
  • 100,966
  • 191
  • 140
  • 197
  • Note that using `Future.exception` is probably faster than throwing and catching an exception. - Idealñly you should never throw exceptions yourself, but rather lift failed values inside the **Future** with combinators like `exception`. – Luis Miguel Mejía Suárez Nov 25 '20 at 13:45
  • @LuisMiguelMejíaSuárez thanks, would you have any pointers/references on why it is faster? – Stefan Pavikevik Nov 26 '20 at 01:12
  • Because throwing and catching an exception is slow, whereas just returning a value is just as fast as retuning any other value. See: https://mattwarren.org/2016/12/20/Why-Exceptions-should-be-Exceptional/ – Luis Miguel Mejía Suárez Nov 26 '20 at 01:29
  • @LuisMiguelMejíaSuárez ah, that makes sense.. since Future has to catch an exception. thanks for the link – Stefan Pavikevik Nov 26 '20 at 01:33

2 Answers2

3

From a theoretical point of view, if we take away the exceptions part (they cannot be reasoned about using category theory anyway), then those two operations are completely identical as long as your construct of choice (in your case Twitter Future) forms a valid monad.

I don't want to go into length over these concepts, so I'm just going to present the laws directly (using Scala Future):

import scala.concurrent.ExecutionContext.Implicits.global

// Functor identity law
Future(42).map(x => x) == Future(42)

// Monad left-identity law
val f = (x: Int) => Future(x)
Future(42).flatMap(f) == f(42) 

// combining those two, since every Monad is also a Functor, we get:
Future(42).map(x => x) == Future(42).flatMap(x => Future(x))

// and if we now generalise identity into any function:
Future(42).map(x => x + 20) == Future(42).flatMap(x => Future(x + 20))

So yes, as you already hinted, those two approaches are identical.

However, there are three comments that I have on this, given that we are including exceptions into the mix:

  1. Be careful - when it comes to throwing exceptions, Scala Future (probably Twitter too) violates the left-identity law on purpose, in order to trade it off for some extra safety.

Example:

import scala.concurrent.ExecutionContext.Implicits.global

def sneakyFuture = {
  throw new Exception("boom!")
  Future(42)
}

val f1 = Future(42).flatMap(_ => sneakyFuture)
// Future(Failure(java.lang.Exception: boom!))

val f2 = sneakyFuture
// Exception in thread "main" java.lang.Exception: boom!
  1. As @randbw mentioned, throwing exceptions is not idiomatic to FP and it violates principles such as purity of functions and referential transparency of values.

Scala and Twitter Future make it easy for you to just throw an exception - as long as it happens in a Future context, exception will not bubble up, but instead cause that Future to fail. However, that doesn't mean that literally throwing them around in your code should be permitted, because it ruins the structure of your programs (similarly to how GOTO statements do it, or break statements in loops, etc.).

Preferred practice is to always evaluate every code path into a value instead of throwing bombs around, which is why it's better to flatMap into a (failed) Future than to map into some code that throws a bomb.

  1. Keep in mind referential transparency.

If you use map instead of flatMap and someone takes the code from the map and extracts it out into a function, then you're safer if this function returns a Future, otherwise someone might run it outside of Future context.

Example:

import scala.concurrent.ExecutionContext.Implicits.global

Future(42).map(x => {
  // this should be done inside a Future
  x + 1
})

This is fine. But after completely valid refactoring (which utilizes the rule of referential transparency), your codfe becomes this:

def f(x: Int) =  {
  // this should be done inside a Future
  x + 1
}
Future(42).map(x => f(x))

And you will run into problems if someone calls f directly. It's much safer to wrap the code into a Future and flatMap on it.

Of course, you could argue that even when using flatMap someone could rip out the f from .flatMap(x => Future(f(x)), but it's not that likely. On the other hand, simply extracting the response processing logic into a separate function fits perfectly with the functional programming's idea of composing small functions into bigger ones, and it's likely to happen.

slouc
  • 9,508
  • 3
  • 16
  • 41
  • This makes sense, although I was not super sure whether Twitter is following FP principles here (they aren't at some others places). – Stefan Pavikevik Nov 26 '20 at 01:17
  • The project I am working on is not really pure FP by any means, and it many places it throws around exceptions (I sometimes I wish I could just change all of that :D) so there was this discussion about which one should we prefer (throwing vs. Future.exception) which is why I posted this question. – Stefan Pavikevik Nov 26 '20 at 01:19
1

From my understanding of FP, exceptions are not thrown. This would be, as you said, a side-effect. Exceptions are instead values that are handled at some point in the execution of the program.

Cats (and i'm sure other libraries, too) employs this technique too (https://github.com/typelevel/cats/blob/master/core/src/main/scala/cats/ApplicativeError.scala).

Therefore, the flatMap call allows the exception to be contained within a satisfied Future here and handled at a later point in the program's execution where other exception value handling may also occur.

randbw
  • 490
  • 5
  • 13