3

Okay, here is my setup:

  • Platform: AWS
  • Monitoring: DataDog
  • Metric: system.disk.in_use

Question: So I am running Ubuntu 18.04LTS instances and as time goes on, it seems to spawn additional devices periodically:

device:/dev/loop1, /dev/loop2 and so on.

When I first spun up these instances, there were only 3 /dev/loop(1-3) devices, however, over time, a /dev/loop4 showed up and our drive space alert paged me since these are 100% utilized when created.

So, I have to go into each of the monitors (one per environment) and add an exclusion for the new /dev/loop4, but I cannot set the exclusion until it has been created by at least one of the monitored instances.

Is there a way in DataDog that you can just add a blanket exclusion like:

device:/dev/loop*?

I have been combing through documentation and have not been able to find anything, so I thought I would ask here.

DC.Skells
  • 830
  • 1
  • 8
  • 18

2 Answers2

4

You can use ! and *, like in:

avg:system.disk.in_use{!device:/dev/loop*} by {host,device}

source

Tobias Sette
  • 185
  • 12
3

I solved this by adding 'squashfs' to the list of filesystem types to be ignored by the datadog agent.

Create a file /etc/datadog-agent/conf.d/disk.d/conf.yaml:

init_config:
    file_system_global_blacklist:
      - iso9660$
      - squashfs

instances:
  - use_mount: false

Restart datadog agent (systemctl restart datadog-agent).

rossigee
  • 51
  • 4