1

I am trying to run spark-ec2-branch-2 script for creating a cluster in Ohio. I need to create a cluster in Ohio because Ohio is one of the regions that allows VPC peering. 

 ./spark-ec2 --key-pair=ohio --identity-file=ohio.pem 
--region=us-east-2 --zone=us-east-2c --vpc-id=vpc-1906ca71 --subnet-id=subnet-26e2ae6b
--instance-type=t2.micro -s 1 launch ohio_cluster

I have set up a cluster in other regions using the similar script. As far as I know ohio is us-east-2 (link). But I am getting the following error:

Setting up security groups... 
Traceback (most recent call last):                                                                                        
File "./spark_ec2.py", line 1573, in <module>                                                                        
main()                                                                                                                
File "./spark_ec2.py", line 1565, in main                                                                                 
real_main()                                                                                                           
File "./spark_ec2.py", line 1394, in real_main                                                                            
(master_nodes, slave_nodes) = launch_cluster(conn, opts, cluster_name)                                                
File "./spark_ec2.py", line 523, in launch_cluster                                                                        
master_group = get_or_make_group(conn, cluster_name + "-master", 
opts.vpc_id)
File "./spark_ec2.py", line 371, in get_or_make_group                                                                     
groups = conn.get_all_security_groups()                                                                             
AttributeError: 'NoneType' object has no attribute 'get_all_security_groups'

I guess the script cannot set up a connection in this region, or cannot find this region. But I am failing to understand why. In any case, if anyone can guide and help me to setup a spark cluster using script in Ohio region I will be very grateful. Thanks in advance.

user3086871
  • 671
  • 3
  • 7
  • 25
  • You didn't give the important information. How are you creating `conn`? Show us the code – helloV Jan 27 '18 at 17:53
  • oh my bad! I should have made myself clear. I am using the script from amplab : https://github.com/amplab/spark-ec2 – user3086871 Jan 27 '18 at 18:08
  • Any updates on this? I am having the same problem. Did the same script work for other regions? I do not need to stick to Ohio, so maybe I can dodge the problem. – W7GVR Jun 03 '18 at 00:13
  • Changing to a different region (us-west-2) worked for me. Problem may be region specific... – W7GVR Jun 03 '18 at 00:35

0 Answers0