0

I am trying to run a spark job via Mesos

it throws an exception

WARN MesosExternalShuffleClient: Unable to register app 6c1b7274-960f-47ef-9fa7-1dd06b05d4f1-0010

with external shuffle service.

Please manually remove shuffle data after driver exit

Error:java.lang.RuntimeException: java.lang.UnsupportedOperationException: Unexpected message: org.apache.spark.network.shuffle.protocol.mesos.RegisterDriver@88d86a24 "

and the logs in Stderr as

ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM INFO DiskBlockManager: Shutdown hook called E0911 05:32:34.711486 6619 process.cpp:951] Failed to accept socket: future discarded**

In Spark-Defaults.conf

spark.mesos.coarse          true
spark.network.timeout           3600s
spark.shuffle.io.connectionTimeout  3600s

Who is killing my application..?

iamsankalp89
  • 4,607
  • 2
  • 15
  • 36
Abhishek
  • 25
  • 5
  • It's probably not a problem with future discarded https://stackoverflow.com/a/46123514/1387612 – janisz Sep 11 '17 at 15:40
  • I am trying to [set up Mesos cluster](http://mesos.apache.org/gettingstarted/) and when I do `make check` I get 8 test fails, first one being this. – Globox Sep 11 '17 at 23:16
  • `./bin/mesos-tests.sh --gtest_filter="CommandExecutorCheckTest.HTTPCheckDelivered" --verbose ` Test mentioned in gtest_filter fails! – Globox Sep 11 '17 at 23:17
  • It looks like this tests are flaky, see [MESOS-7440](https://issues.apache.org/jira/browse/MESOS-7440) – janisz Sep 18 '17 at 15:07

0 Answers0