1

Up until now I've basically used spot instances as cheaper on-demand instances, but the changes made since last November make it harder to predict interruption (and thus choose instances that won't get interrupted), so I'm trying to figure out how to use spot instances in the way they they are intended to be used.

Luckily, my jobs are quite fault-tolerant -- this one is large-scale geoprocessing. My problem is getting by startup shell script to work. I'm not sure if it is an issue with the shell script itself, an issue with how I'm interfacing with AWS, or some other issue.

Here is my bash script:

#!/bin/bash
# make a directory
mkdir /home/ubuntu/MACA
# attach the EBS volume that has all of my data and scripts
aws ec2 attach-volume --volume-id vol-6969420420 --instance-id $(curl http://169.254.169.254/latest/meta-data/instance-id) --device /dev/xvdf
# mount it
mount /dev/xvdf /home/ubuntu/MACA
# remove cruft left over from interrupted runs
rm /home/ubuntu/MACA/tmp/+macav2metdata_*
# run my R script that implements my job
Rscript --verbose /home/ubuntu/MACA/tmp/process_maca_v2.r

I don't do a lot of shell scripting -- is something amiss here? I tell AWS to run it via the console: enter image description here

When I go and SSH in, nothing is happening. Here is the relevant part of the /var/log/cloud-init-output.log:

Cloud-init v. 17.1 running 'modules:config' at Wed, 14 Feb 2018 22:38:25 +0000. Up 40.96 seconds.
Cloud-init v. 17.1 running 'modules:final' at Wed, 14 Feb 2018 22:38:32 +0000. Up 48.23 seconds.
2018-02-14 22:38:32,757 - util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-001 [-]
2018-02-14 22:38:32,859 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
2018-02-14 22:38:32,865 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py'>) failed
Cloud-init v. 17.1 finished at Wed, 14 Feb 2018 22:38:33 +0000. Datasource DataSourceEc2Local.  Up 48.77 seconds

Some sort of python error? I don't know how to parse this. Would appreciate help getting this running. Please let me know if I haven't provided enough info.

generic_user
  • 3,430
  • 3
  • 32
  • 56
  • 1
    Are you using a custom AMI ? If you are, check [this](https://stackoverflow.com/questions/27086639/user-data-scripts-is-not-running-on-my-custom-ami-but-working-in-standard-amazo) out. The part about clearing the /var/lib/cloud directory worked for me. – Sanchit Anand Feb 14 '18 at 22:52
  • Yes I am! Will chase this down tomorrow – generic_user Feb 14 '18 at 22:58

1 Answers1

0

Since you are creating a custom AMI from a running instance, the cloud-init code has already been executed. Before creating an AMI, run 

rm -rf /var/lib/cloud/*

to clear all the existing bootstrap data. After this, create your image and the user data scripts should work fine. Alternatively, use 

#cloud-boothook

at the start of your script to force it to execute, but note that it is not guaranteed to execute just once.

Sanchit Anand
  • 195
  • 2
  • 11