How to monitor EC2 instances by memory?

Question

Using Cloudwatch you can monitor your EC2 instances by several criteria, such as network usage, CPU usage, and so on…

Unfortunately, there is no metric for memory consumption. First of all, just out of curiosity, I would like to know, why? Can anybody explain why it is possible to, e.g., monitor CPU usage, but not memory usage? At least to me, that's not obvious.

And then, my actual question: Okay, given that Cloudwatch doesn't allow monitoring the EC2 instances' memory usage - what is the alternative? How should I setup an alarm if, e.g. > 80% of the memory of an instance is being used?

It's been 5 years since the question and AWS still doesn't have an easy and straight-forward way to monitor memory usage of EC2 instances :( What a shame. — Sarang, Jul 30 '22 at 19:59

score 41 · Accepted Answer · answered Feb 18 '17 at 15:44

41

Memory and Disk specific statistics require AWS to monitor at the OS level rather than the host level, so that is why they leave it out by default. It will probably be added at some point but since it has been on the wish list for about 7 years, we can assume it's a very low priority item.

The recommended way to monitor memory usage is to create a custom Cloudwatch metric by using your own monitoring scripts on your instances. AWS have published documentation on how to achieve this on Linux instances with a set of (unsupported) scripts.

Once your instances are publishing the custom metrics, you will be able to attach alarms to them in CloudWatch.

answered Feb 18 '17 at 15:44

DF_

3,743
25
34

1

CloudWatch recently released a collectd plugin that can publish collectd metrics to Cloudwatch and collectd can read memory usage (among many other things) https://aws.amazon.com/blogs/aws/new-cloudwatch-plugin-for-collectd/ – Dejan Peretin Feb 18 '17 at 21:42
21

Arguably, about every other third-party solution does this out of the box. Oh, and Azure and GCP respective monitoring agents also do this. AWS just doesn't give a single F* about this issue. – solr Dec 24 '20 at 14:14

score 31 · Answer 2 · answered Aug 14 '19 at 18:26

31

Well, now the new CloudWatch agent can collect metrics like memory and disk usage, see the docs.

With this you can monitor this kind of metrics, but you will need to install and configure the agent in the instances.

answered Aug 14 '19 at 18:26

dpolicastro

1,379
1
14
22

score 5 · Answer 3 · edited Apr 08 '23 at 20:44

Well by default the AWS Ec2 does not provide memory metrics. So we have to install the cloud watch agent separately and then get the metrics

Steps:

Download the cloud watch agent from s3

sudo wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
Install the cloud watch agent

sudo rpm -U ./amazon-cloudwatch-agent.rpm
We need to write a config file in which we will specify the metrics we need to put.

sudo vi /opt/aws/amazon-cloudwatch-agent/bin/config.json

you need to insert this inside the config file

{
    "metrics": {
        "metrics_collected": {
            "mem": {
                "measurement": [
                    "mem_used_percent"
                ],
                "metrics_collection_interval": 30
            }
        }
    }
}

Then after specifying the metric then we just need to start the cloud watch agent

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s
You can go and test out if the metric is coming or not in the cloud watch page

Check:

Cloudwatch console --> All metrics --> under custom namespace --> CW Agent --> host --> ip of the instance server

Now you can view the memory usage and monitor the uage

AWS Documentation: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/download-cloudwatch-agent-commandline.html

The custom namespace is **CWAgent**. If you don't see it, make sure you have added the Cloudwatch policy to the running instance ([docs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-iam-roles-for-cloudwatch-agent.html)). You can also check the logs for any issue you may have: ```/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log``` or ```/var/log/amazon/amazon-cloudwatch-agent/amazon-cloudwatch-agent.log``` — Edenshaw, Feb 27 '23 at 16:40

How to monitor EC2 instances by memory?

3 Answers3

Linked