1

Context: I want to write scalding job(hadoop) to crawl pages and I want to set timeout on url extraction(without timeout on URLConnection, I want generic solution for other timeout cases) i.e. map function.

I'm think about futures which are killed after some timout with all resources released since it is memory critical code. Not sure what to use in scala API.

yura
  • 14,489
  • 21
  • 77
  • 126
  • 1
    You can use akka futures, but they throw an exception on timeout, which may or may not be what you're looking for http://doc.akka.io/docs/akka/2.0.1/scala/futures.html – Noah Jun 15 '12 at 21:08
  • I would prefer build in scala API, since any new dependency is a pain in hadoop jar. – yura Jun 16 '12 at 00:32
  • It's not a duplicate but I asked a similar question: http://stackoverflow.com/questions/7651293/computation-with-time-limit Perhaps one of the answer will help you. – paradigmatic Jun 16 '12 at 07:03

1 Answers1

0

While Akka was suggested--and is superior to the following solution--Scala does have its own, built in Actor model much like Akka which can do you want you want. Examples can be found here:

http://www.scala-lang.org/node/242

You'll probably want either reactWithin or receiveWithin, the documentation for which can be found here:

http://www.scala-lang.org/api/current/scala/actors/TIMEOUT$.html

Jeff Van Vranken
  • 58
  • 6