4

I'm using Spark Job Server to run a Spark job and it works perfectly. But when I try to execute a big job (needs more than 40 sec) I get this error: 

The server was not able to produce a timely response to your request.

Is there some configuration required in order to wait for the server answer? What should I do?

Thank you

vatsal mevada
  • 5,148
  • 7
  • 39
  • 68
Amine CHERIFI
  • 1,155
  • 2
  • 20
  • 35

3 Answers3

7

There are several different timeouts that you can hit with sync job submissions. Yours is coming from Spray-can server. You can configure it through your conf file:

spray.can.server {
      idle-timeout = <set desired timeout>
      request-timeout = <set desired timeout>
}

You can set both values to infinite to disable these timeouts at all.

In general, there are at least 3 common different timeouts that I've observed and that can return your request before job completion:

  1. Akka-based ask timeout. Will return a JSON response with "ERROR" status, if sync job was not completed before X sec. Default in SJS is 10sec but you can overwrite it by passing timeout=Y argument to your POST /jobs request.
  2. Spray-can server timeouts: idle-timeout, request-timeout. Will return default Spray response, making it harder to catch. They default to 60 and 40 secs respectively.
  3. Yet another timeout may come from your client REST library... Some of them configure their defaults to disable any timeout on this layer, some of them may take into account the server's timeouts and configure their own behavior according to that... For example, C++ REST SDK library kicks in WinHTTP timeout of 30 secs once you disable timeout #2 above, with #2 in place it will wait for 40 sec...
Oleg Shirokikh
  • 3,447
  • 4
  • 33
  • 61
1

In your Rest call, place(sync=false) at the end of the URL. Similar to http://server:8090/jobs?classPath=.... &sync=false at the end. It will start the job on the server and provide you a JobId.

The JobId can then be used to get the results:

For example: http://server:8090/jobs/b3b46a27-f711-469w-be09-4942006896b5

If that job is not finished, it will indicate the status as RUNNING. If it is finished, it will give you a status of FINISHED and the results.

user422930
  • 259
  • 1
  • 5
  • 15
0

I had the same problem. The error message seems to be generic. When I check logs I saw:

Oops, there's an AbstractMethodError... maybe you compiled your code with an older version of SJS? here's the exception:

And when I change the version to 0.7 it worked. SO probably this error sometimes means there is an error.You have to check logs.

mcelikkaya
  • 315
  • 4
  • 13