0

Possible Duplicate:
Re-use Amazon Elastic MapReduce instance

Can I keep a launched EMR cluster running and keep submitting new jobs to it until I am done (say after a couple of days) and then shut down the cluster or do I have to lanuch my own cluster in EC2 to do so?

Community
  • 1
  • 1
iCode
  • 4,308
  • 10
  • 44
  • 77

1 Answers1

1

Yes.

In particular, I use the CLI client. Here is a snippet from one of my scripts:

JOBFLOW_ID=`elastic-mapreduce --create --alive --name cluster --num-instances $
COUNT --hadoop-version 0.20.205 --ami-version 2.0 --plain-output --debug --log-uri $LOG_
URI`

echo "Launched --jobflow $JOBFLOW_ID"

elastic-mapreduce -j $JOBFLOW_ID --ssh

You could terminate with -j --term, issue cli commands to the master using --ssh. There is even a page in the DevGuide (which you should really read it) about adding more than 256 steps (done on a already launched instance, mind that), so its really not a huge concern.

aldrinleal
  • 3,559
  • 26
  • 33
  • Hey is there a way where I can reuse the instance using the aws sdk where I can set jobflowid (I thought there might be a setjobflowid function for some class in aws sdk). This is my question : http://stackoverflow.com/questions/25994449/re-use-amazon-elastic-mapreduce-instance-using-aws-java-sdk and I get a downvote for no apparent reason. – Akshay Hazari Sep 24 '14 at 04:36