-1

With the below configuration for AWS cloud watch:

awslogs.conf

[/var/log/messages]
datetime_format = %b %d %H:%M:%S
file = /var/log/messages
buffer_duration = 2500
log_group_name = /var/log/messages
log_stream_name = {cluster}{instance_id}

and below script used with --userdata option for aws ec2 command:

userdata.sh

# Above agentlogs.conf file is copied to /etc/awslogs/awslogs.conf in AWS EC2 instance


# Configure cloudwatch  config file
cat > /etc/cloudwatch-logs.ini <<EOF
[/var/log/messages]
datetime_format = %b %d %H:%M:%S
file = /var/log/messages
buffer_duration = 2500
log_stream_name = {cluster}{instance_id}
initial_position = start_of_file
log_group_name = /var/log/messages
EOF

an EC2 is launched from this script(running outside EC2):

spin_up_ec2.sh

# Using AWS CLI,  we spin up EC2 instance using userdata.sh, 
# Using metadata service How to read values of {cluster} & {instance_id} syntax, shown above:

aws logs describe-log-streams --log-group-name /var/log/messages --log-stream-name-prefix <grab_cluster_name_value><grab_instance_id_value> --region us-east-1

spin_up_ec2.sh is sitting outside EC2, within same VPC, in different subnet. So, am not sure, how to avail EC2 metadata service?

EC2 is running in private subnet.

{cluster} value would be something like clust1

{instance_id} value would be something like i-1a52627268bc

1)

How can a shell script(spin_up_ec2.sh) client talk to EC2 metadata service, to retrieve values of {cluster} & {instance_id}?

2)

Does launching EC2 in public subnet, help? To talk to metadata service

overexchange
  • 15,768
  • 30
  • 152
  • 347
  • have you tried printing the value for instance_id before you use it in your script? – Prabhakar Reddy Jan 09 '20 at 05:17
  • @bdcloud does this work? In my userdata script... `echo {instance_id}` because {...} Is not shell syntax – overexchange Jan 09 '20 at 06:02
  • @bdcloud what value do you think `{instance_id}` should display? – overexchange Jan 09 '20 at 06:14
  • How are you parsing this instance id from user data? – Prabhakar Reddy Jan 09 '20 at 06:16
  • @bdcloud Query updated with the way we are trying to read {instance_id} – overexchange Jan 09 '20 at 06:46
  • Does this answer your question? [How do I grab an INI value within a shell script?](https://stackoverflow.com/questions/6318809/how-do-i-grab-an-ini-value-within-a-shell-script) – KamilCuk Jan 09 '20 at 13:28
  • @KamilCuk No, because I don not want `{instance_id}` literally, but the value of it. AWS cloudwatch will replace it value something like `i-1a536218638bc` – overexchange Jan 09 '20 at 13:44
  • That's what instance metadata is for :) – favoretti Jan 09 '20 at 13:50
  • @favoretti `http://169.254.169.254/latest/meta-data/`. That is correct!!!! but how would `spin_up_ec2.sh`( running outside that EC2 but within same AWS account) talk to EC2 instance metadata service, where EC2 is in private subnet with it's private IP(no publicIP). This http request cannot work, Isn't it? Basically Jenkins job within same AWS account is running `spin_up_ec2.sh` script. – overexchange Jan 09 '20 at 13:58
  • I mean `http://169.254.169.254/latest/meta-data/` will work, if requested from within EC2 – overexchange Jan 09 '20 at 14:03
  • Ah, you want to run that script from outside of that EC2.. well, then you're out of luck. You can use awscli and get instance ID based on tags or something like that.. – favoretti Jan 09 '20 at 14:41
  • @favoretti can you suggest me an approach on this? I need cluster and instance_id name – overexchange Jan 10 '20 at 10:25
  • One approach would be to tag your instances in a way that you can uniquely filter them by using those tags and fetch instance id that way. – favoretti Jan 11 '20 at 16:39
  • @favoretti What is the command to get instance_id? for fetching based on tags.. – overexchange Jan 11 '20 at 19:37

1 Answers1

2

The Amazon EC2 instance metadata is not available outside of an instance.

You could make API calls to AWS services to obtain similar information (eg retrieve the subnet in which an EC2 instance is located).

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470