5

I want my compute engine VM to show memory usage metrics in the console, I went to this page and install Ops-Agents, restart the service and went to the VM observability section, but still saw a message that the agent is not installed (in the memory usage metric): enter image description here I thought maybe by default the memory usage is not installed (it's not mentioned anywhere, just a guess) and I need to modify the config. I went to this docs and added this code to /etc/google-cloud-ops-agent/config.yaml:

metrics:
  receivers:
    agent.googleapis.com/memory/bytes_used:
      type: hostmetrics
      collection_interval: 1m

According to the docs, this config will be merged with the built-in configuration when the agent restarts. I restarted the agent service, went back to the dashboard but still it shows the message "Requires Ops Agent". I don't know what I'm doing wrong, the documentations are really poor for that topic IMO, I couldn't find any example on how to turn on memory usage metrics.

EDIT Running sudo systemctl status google-cloud-ops-agent"*" I can see this error message:

otelopscol[2763]: 2022-05-02T14:07:02.780Z#011error#011collector@v0.26.1-0.20220307211504-dc45061a44f9/metrics.go:235#011could not export time series to GCM#011{"error": "rpc error: code = InvalidArgument desc = Name must begin with '{resource_container_type}/{resource_container_id}', got: projects/"}

EDIT2 If I click INSTALL via the console, I see this installation instructions:

:> agents_to_install.csv && \
echo '"projects/<project>/zones/europe-west1-b/instances/<instance>","[{""type"":""ops-agent""}]"' >> agents_to_install.csv && \
curl -sSO https://dl.google.com/cloudagents/mass-provision-google-cloud-ops-agents.py && \
python3 mass-provision-google-cloud-ops-agents.py --file agents_to_install.csv

It's differente from the one here: https://cloud.google.com/monitoring/agent/monitoring/installation#joint-install

curl -sSO https://dl.google.com/cloudagents/add-monitoring-agent-repo.sh
sudo bash add-monitoring-agent-repo.sh --also-install

Not sure what installed what, tried both.

Avishay28
  • 2,288
  • 4
  • 25
  • 47
  • In your user configuration (which gets merged with the built-in configuration) you define a receiver, and I am not sure if the name 'agent.googleapis.com/memory/bytes_used' leads to the error you see when you restart the service. – Dirk R Jun 15 '22 at 13:57
  • If you remove the user configuration and restart the service, does it show an error? What does the syslog tell you now? It should show the default (builtin) configuration. From my experience, the builtin configuration does already cover the required metric, as it uses "hostmetrics" as receiver with an empty filter in the processors part, and combines that in the services part. Also, the Google Cloud Console tells you that the ops agent is not installed, at all. Please take a look at https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent/troubleshooting – Dirk R Jun 15 '22 at 14:03

2 Answers2

0

Regarding your questions “I couldn't find any example on how to turn on memory usage metrics” and “Is it installed but the configurations need to be modified for the memory usage metrics?” the answer is yes, you need to customize which group or groups of metrics to enable as specified here. The metric type strings must be prefixed with agent.googleapis.com/agent/. For memory metrics, the examples are:

agent.googleapis.com/agent/memory_usage

agent.googleapis.com/agent/memory_utilization

That prefix has been omitted from the entries in the table that I’m sharing here.

Now, you need to select the setting based on the target VM that you need to get metrics from, for example, Linux only:

agent.googleapis.com/memory/usage

Also, you can play with other options, changing the final criteria, for example:

agent.googleapis.com/memory/bytes_used

Ensure that you didn’t miss anything regarding the agent’s installation, follow these instructions to install it from the CLI. Then go to:

  • Resources -> Instances: You should see your VM instance.

  • Click on your instance -> click on Agent -> Scroll down and you see your memory and your swap usage.

Finally, you can follow this troubleshooting guide for Ops Agent issues, and these threads for more empirical cases and solutions Memory Usage Monitoring in GCP Compute Engine and No metric found.

Nestor Daniel Ortega Perez
  • 1,046
  • 3
  • 10
  • "'Is it installed but the configurations need to be modified for the memory usage metrics?'" the answer is yes" - apparently no, I just started a fresh small instance, installed the agent and I can see memory usage metrics with no custom configuration. The only difference between the instance I had error and the new one is that the one with error have "Legacy Agent" as show in the monitor vm instances section. I'm not sure why, I installed in both instances the agent with the same installation guide (the one you provided) – Avishay28 May 03 '22 at 08:00
  • So, is it now working in both instances? If you are still facing the issue in the original instance, can you follow the installation guide from the **CLI** that I shared with you and the troubleshooting steps? – Nestor Daniel Ortega Perez May 03 '22 at 13:52
  • Thanks for the help. The original one is not working, I've tried the troubleshoot guide and the only error I see in the logs when running `sudo systemctl status google-cloud-ops-agent"*"` is `could not export time series to GCM {"error": "rpc error: code = InvalidArgument desc = Name must begin with '{resource_container_type}/{resource_container_id}', got: projects/"}` which triggers from `collector@v0.26.1-0.20220307211504-dc45061a44f9/metrics.go:235`. I guess this is the cause of the issue but I can't find a way to resolve this. – Avishay28 May 03 '22 at 15:13
0

I had this same concern when trying to show memory metrics in GCP compute virtual machines.

This script worked for me:

curl -sSO https://dl.google.com/cloudagents/add-google-cloud-ops-agent-repo.sh
sudo bash add-google-cloud-ops-agent-repo.sh --also-install

References:

Promise Preston
  • 24,334
  • 12
  • 145
  • 143