19

I am creating an EMR cluster, and using jupyter notebook to run some spark tasks. My tasks die after approximately 1 hour of execution, and the error is:

An error was encountered:
Invalid status code '400' from https://xxx.xx.x.xxx:18888/sessions/0/statements/20 with error payload: "requirement failed: Session isn't active."

My understanding is that it is related to the Livy config livy.server.session.timeout, but I don't know how I can set it in the bootstrap of the cluster (I need to do it in the bootstrap because the cluster is created with no ssh access)

Thanks a lot in advance

bill
  • 293
  • 2
  • 6
  • 17

3 Answers3

32

On EMR, livy-conf is the classification for the properties for livy's livy.conf file, so when creating an EMR cluster, choose advanced options with Livy as an application chosen to install, please pass this EMR configuration in the Enter Configuration field.

[{'classification': 'livy-conf','Properties': {'livy.server.session.timeout':'5h'}}]
  • On EMR, Livy binary is located at /etc/livy/, and so the config file is at /etc/livy/conf/livy.conf

To verify this,

  • Create an EMR cluster with a known ec2 key-pair, Livy and above config
  • Using the ec2 key-pair, login to the EC2 Master node associated with the cluster ssh -i some-ec2-key-pair.pem hadoop@ec2-00-00-00-0.ca-region-n.compute.amazonaws.com
  • Navigate to /etc/livy/conf, vim livy.conf & see the updated value of livy.server.session.timeout
N_C
  • 952
  • 8
  • 17
  • 5
    Replace 'classification' with 'Classification'. – jaycode Feb 04 '20 at 07:21
  • how can we change livy.rsc.launcher.port.range property in livy-client.conf ? unable to do so, classification "livy-client" or "livy-client-conf" is not working. – manmeet Mar 04 '20 at 08:29
  • 1
    Hi @Naveen, you can add this link to your answer https://aws.amazon.com/premiumsupport/knowledge-center/emr-session-not-found-http-request-error/ – Benzi Jul 08 '21 at 09:00
6

If you don't want the Livy session to go down at all, then set the property livy.server.session.timeout-check to false in /etc/livy/conf/livy.conf.

amruth
  • 81
  • 1
  • 4
  • 1
    Using AWS console EMR page for my cluster, I'm able to edit table and add configuration for all instance groups, using `livy-conf` and `livy.server.session.timeout-check`, `false`. The master and core go into 'reconfiguring' mode after. – Merlin Nov 30 '20 at 14:40
  • @Merlin, my Master and Core went into "Reconfiguring" status for a while, then the master became "Suspended" and the core "Running". There's a warning that says "Master Instance Group: Configuration reversion failed, arrest instance group." Is that what happened to you? – puifais Jul 06 '21 at 00:03
3

Another way to do that if you don’t want to recreate the cluster is: go to /etc/livy/conf/livy.conf and set the livy.server.session.timeout property to the value you would like. After that, run sudo restart livy-server to make the configuration applied.

Jeremy
  • 155
  • 1
  • 3