1

I am getting the following exception in my Spark application:- 

ERROR ApplicationMaster: Uncaught exception: 
java.util.concurrent.TimeoutException: Futures timed out after [100000 milliseconds]
    at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
    at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
    at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:201)
    at org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:401)
    at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:254)
    at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$main$1.apply$mcV$sp(ApplicationMaster.scala:764)
    at org.apache.spark.deploy.SparkHadoopUtil$$anon$2.run(SparkHadoopUtil.scala:67)
    at org.apache.spark.deploy.SparkHadoopUtil$$anon$2.run(SparkHadoopUtil.scala:66)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
    at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:66)
    at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:762)
    at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)

This came after all my tasks had ran successfully. The tasks ran for about 70s. Anyone has ideas as to why such an error is coming up. The ques here:- What are possible reasons for receiving TimeoutException: Futures timed out after [n seconds] when working with Spark, looks like some timeout happening because of SQL. Here, however it look like there is some network related timeout.

Sanchay
  • 1,053
  • 1
  • 16
  • 33
  • The ques here:- https://stackoverflow.com/questions/40474057/what-are-possible-reasons-for-receiving-timeoutexception-futures-timed-out-afte looks like some timeout happening because of SQL. Here, however it look like there is some network related timeout. – Sanchay Nov 27 '17 at 13:28
  • 1
    The issue was I still had .setMaster("local") in my code, because of which I was not able to use the cluster functionality. Once I removed that, the job ran successfully. – Sanchay Nov 28 '17 at 07:58

0 Answers0