I added simple rule to get free disk space from node exporter, after adding rule file to
prometheus.yml
file, prometheus is not starting, could be some prob with rule file.
# my global config
global:
scrape_interval: 60s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 60s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout is set to the global default (10s).
# Alertmanager configuration
alerting:
alertmanagers:
- static_configs:
- targets:
- alertmanager:9093
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
- 'low-disk-space.rules'
# - 'sample-rules.yml'
# - "first_rules.yml"
# - "second_rules.yml"
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
#- job_name: 'prometheus'
# metrics_path defaults to '/metrics'
# scheme defaults to 'http'.
#static_configs:
#- targets: ['localhost:9090']
- job_name: 'machine'
static_configs:
- targets: ['localhost:9100']
my rule file looks like below:
groups:
- name: disk_usage
interval: 5s
rules:
- record: node_exporter:node_filesystem_free:fs_used_percents
expr: 100 - 100 * ( node_filesystem_free{mountpoint="/"} / node_filesystem_size{mountpoint="/"} )
- name: alerting_rules
rules:
- alert: DiskSpace10%Free
expr: node_exporter:node_filesystem_free:fs_used_percents >= 90
labels:
severity: moderate
annotations:
summary: "Instance {{ $labels.instance }} is low on disk space"
description: "{{ $labels.instance }} has only {{ $value }}% free."
prometheus version: 2.3.2 posted another question regards to validation of rule file here how to install latest version of prometheus/promtool in ubuntu?