User Data in AWS CloudFormation for an EC2 Instance through a custom AMI not working

Question

I am unable to figure out why the echo commands in the shell script for EC2 Instance User Data creation in AWS CloudFormation is not running. While the "sudo systemctl" commands are working.

I did try "sudo echo" as well, which did not work.

There are no errors. The python flask application which is set to run on bootup using "sudo systemctl" command is working fine. But there is no .env file created.

I am using the free-tier Amazon Linux image from the AMI catalog:

Amazon Linux 2 AMI (HVM) - Kernel 5.10, SSD Volume Type ami-0c02fb55956c7d316 (64-bit (x86))

ImageId is a reference to the custom AMI I created for the python flask application based on the AMI mentioned above.

~/webapp/release is my working directory

# Create EC2 Instance
  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: !Ref ImageId
      InstanceType: t2.micro
      KeyName: csye6225
      UserData:
        Fn::Base64: 
          !Sub |
              #!/bin/bash
              cd ~/webapp/release
              echo "DB_HOST=\"${DatabaseInstance.Endpoint.Address}\"" >> .env
              echo "DB_PORT=\"${DatabaseInstance.Endpoint.Port}\"" >> .env
              echo "DB_DATABASE=\"${DatabaseName}\"" >> .env
              echo "DB_USERNAME=\"${DatabaseUser}\"" >> .env
              echo "DB_PASSWORD=\"${DatabasePassword}\"" >> .env
              echo "FILESYSTEM_DRIVER=\"s3\"" >> .env
              echo "AWS_BUCKET_NAME=\"${S3Bucket}\"" >> .env
              cd /etc/systemd/system
              sudo systemctl daemon-reload
              sudo systemctl enable flaskapp.service
              sudo systemctl start flaskapp.service
              sudo systemctl status flaskapp.service
      BlockDeviceMappings:
        - DeviceName: /dev/xvda
          Ebs:
            VolumeType: "gp2"
            DeleteOnTermination: "true"
            VolumeSize: "20"
      NetworkInterfaces:
        - AssociatePublicIpAddress: "true"
          DeviceIndex: "0"
          SubnetId: !Ref PublicSubnet2
          GroupSet: [!Ref SSHSecurityGroup]
      IamInstanceProfile: !Ref DemoInstanceProfile
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-EC2"

Error Log from /var/log/cloud-init-output.log:

Cloud-init v. 19.3-45.amzn2 running 'modules:final' at Fri, 25 Mar 2022 00:46:29 +0000. Up 19.92 seconds.
Created symlink from /etc/systemd/system/multi-user.targer.wants/flaskapp.service to /etc/systemd/system/flaskapp.service.
⬤ flaskapp.service - Flask App service
    Loaded: loaded (/etc/systemd/system/flaskapp.service; enabled; vendor preset: disabled)
    Active: active (running) since Fri 2022-03-25 UTC; 45ms ago
Main PID: 3382 ((bash))
    CGroup: /system.slice/flaskapp.service
            |__3382 (bash)

"not working" is not specific. What exactly is happening? What errors do you get? How to reproduce your issue? How to verify it does not work, or works? — Marcin, Mar 25 '22 at 00:04
Also your question lacks details. You haven't even provided what os are you using? Which linux exactly? What is `ImageId` or what is `~/webapp/release`? — Marcin, Mar 25 '22 at 00:05
@Marcin updated the question based on your comments, please let me know if you think more details are needed. Thanks! — supersaiyan, Mar 25 '22 at 00:19
Did you check any logs for errors? such as `/var/log/cloud-init-output.log`? Writing "not working" does not provide any useful information on what you are experiencing. — Marcin, Mar 25 '22 at 00:31
@Marcin Just added the logs, please suggest improvements. Thanks. — supersaiyan, Mar 25 '22 at 02:00

score 2 · Answer 1 · answered Mar 25 '22 at 03:36

2

By default user data executes as root. This cd ~/webapp/release will translate to cd /root/webapp/release, resulting in your error.

You have to ensure that your webapp/release is in the /root or that you use absolute paths to your app, e.g.:

cd /home/ec2-user/webapp/release

answered Mar 25 '22 at 03:36

Marcin

215,873
14
235
294

fixed this error, and updated the question. – supersaiyan Apr 05 '22 at 01:38

score 0 · Accepted Answer · answered Apr 05 '22 at 01:48

As @Marcin suggested, fixed the issue with file path in the first line of the user data script. But still the user data script did not execute. Since my AMI was a custom AMI, I had to add below fixes to make it working.

Looked up more resources on this and eventually this solution worked out perfectly.

My userdata script now has below commands:

      #!/bin/bash
      cd /home/ec2-user/webapp
      sudo rm -rf /var/lib/cloud/instance
      sudo rm -rf /var/lib/cloud/instances
      echo -n "" > .env
      echo DB_HOST=${DatabaseInstance.Endpoint.Address} >> .env
      echo DB_PORT=${DatabaseInstance.Endpoint.Port} >> .env
      echo DB_DATABASENAME=${DatabaseName} >> .env
      echo DB_USERNAME=${DatabaseUser} >> .env
      echo DB_PASSWORD=${DatabasePassword} >> .env
      echo FILESYSTEM_DRIVER=s3 >> .env
      echo BUCKET=${S3Bucket} >> .env
      echo FLASK_ENV=development >> .env
      cd /etc/systemd/system
      sudo systemctl daemon-reload
      sudo systemctl enable flaskapp.service
      sudo systemctl start flaskapp.service
      sudo systemctl status flaskapp.service

User Data in AWS CloudFormation for an EC2 Instance through a custom AMI not working

2 Answers2