0
def resolve(url: String): Future[WSResponse] = ws.url(url).withFollowRedirects(true).get()

def validateAllLinks(links: List[String]) = ???

How would you solve the second function?

I have tried traverse, Future.sequence, Await ...

This was my most recent try:

def validateAllLinks(links: List[String]) = links.map(link =>
    Await.result(resolve(link), Duration.create(3, TimeUnit.SECONDS))
)

The problem with this approach:

  1. i get TimeoutExceptions and MaxRedirectException that i don't want to try/catch

  2. I don't think the solution would be concurrent, even if it worked.

Thanks in advance!

Sam Bokai
  • 538
  • 1
  • 5
  • 13

1 Answers1

3

Feel free to use Future.sequence, they will be running in parallel:

def validateAllLinks(links: List[String]): Future[List[WSResponse]] = 
    Future.sequence(links.map(resolve))

If you call Await.result after Future.sequence, it will wait for the parallel operations to complete:

import scala.concurrent.duration._
def validateAllLinks(links: List[String]): List[WSResponse] = 
    Await.result(Future.sequence(links.map(resolve)), 10 seconds)

But please keep in mind that Await.result is considered a bad practice, you should try to return the Future, and let the client decide how to handle it.

A little more detail about why it's a bad practice:

  1. https://monix.io/docs/2x/best-practices/blocking.html

  2. https://github.com/alexandru/scala-best-practices/blob/master/sections/4-concurrency-parallelism.md#45-should-not-block

Feyyaz
  • 3,147
  • 4
  • 35
  • 50
  • Correct, however there is a problem in this approach that it doesn't allow to handle timeouts and errors for individual requests. Probably this link: https://stackoverflow.com/questions/17466889/run-multiple-futures-in-parallel-return-default-value-on-timeout helps. – igorpcholkin Dec 23 '17 at 00:15
  • Right, my solution was to make it concurrent. To handle the timeouts, `ws.url(url).withRequestTimeout(timeout)` should be used in resolve method. And `resolve(..).map(..).recovery(..)` should be used to understand which ones fail and which ones succeed. – Feyyaz Dec 23 '17 at 12:36