Generating a log file if Memory utilisation is more than 90% in Ansible playbook

Question

I have an existing Ansible playbook which checks the Memory utilization of the target machine and based on the output it receives from the target machine it prints CPU utilization is "Okay" or "Not Okay" on the console.

I want to print the output to a custom log file(utilisation) only when the CPU utilisation is more than 90 %

Ansible playbook :

---
- name:  Linux Memory Usage Monitoring
  hosts: all

  tasks:

    - name: 'copy Get-Memory-Utilization.sh script to {{ inventory_hostname }}'
      copy:
        src:  Get-Memory-Utilization.sh
        dest: /tmp
        mode: '0775'

    - name: 'Preparing Memory utilization using script results'
      shell: |
        sh /tmp/Get-Memory-Utilization.sh
      register: memsec


    - name: 'Preparing Memory utilization for 1st sec'
      shell: |
        sh /tmp/Get-Memory-Utilization.sh
      register: mem1sec


    - name: 'Preparing Memory utilization for 2nd sec'
      shell: |
        sh /tmp/Get-Memory-Utilization.sh
      register: mem2sec


    - name: 'Preparing Memory utilization for 3rd sec'
      shell: |
        sh /tmp/Get-Memory-Utilization.sh
      register: mem3sec


    - name: 'Prepare Memory Used percentage if its abnormal'
      shell: |
        sh /tmp/Get-Memory-Utilization.sh
      register: memhigusage
      when: memsec.stdout|int >= 90 or mem1sec.stdout|int >= 90 or mem2sec.stdout|int >= 90 or mem3sec.stdout|int >= 90

    - name: 'Print message if MEMORY utilization become normal'
      debug:
        msg:
          - -------------------------------------------------------
          -  Memory Utilization = ( ( Total - Free ) / Total * 100 ) = {{ memsec.stdout }}%
          - -------------------------------------------------------
      when: memsec.stdout|int < 90 and mem1sec.stdout|int < 90 and mem2sec.stdout|int < 90 and mem3sec.stdout|int < 90

    - name: 'Print message if MEMORY utilization become abnormal'
      debug:
        msg:
           - -------------------------------------------------------
           - Memory Utilization = ( ( Total - Free ) / Total * 100 ) = {{ memhigusage.stdout }}%
           - -------------------------------------------------------
      when: memsec.stdout|int >= 90 or mem1sec.stdout|int >= 90 or mem2sec.stdout|int >= 90 or mem3sec.stdout|int >= 90

Output which i am getting now on the console is for less than 90%(okay), Suppose this output was when I set the threshold value to be 10% and now this output i want in log file instead of console.

TASK [Print message if MEMORY utilization is  normal] *************************************************************************************************************************************************************
ok: [44.203.153.54] => {
    "msg": [
        "-------------------------------------------------------", 
        "Memory Utilization = ( ( Total - Free ) / Total * 100 ) = 13.87%", 
        "-------------------------------------------------------"
    ]
}

TASK [Print message if MEMORY utilization is  abnormal] ***********************************************************************************************************************************************************
skipping: [44.203.153.54] => {}

Answer 1

I recommend heeding the comments given. Nevertheless, general logging may have its justification under some circumstances.

There are mainly two (2) options

• Writing distributed logs, in example on Remote Node(s)
• Writing centralized log, in example on the Control Node

Distributed Logs

It is assumed that the log directory (i.e. /var/log/ansible) exits and permissions are given. Furthermore, gather_facts: true and the execution path is defined (i.e. by ROLE: "{{ playbook_dir.split('/')[2] }}").

- name: "Log applying role {{ ROLE }} with tags {{ ansible_run_tags }}"
  lineinfile:
    path: "/var/log/ansible/{{ ROLE }}/last.{{ ansible_date_time.year }}{{ ansible_date_time.month }}{{ ansible_date_time.day }}.log"
    create: yes
    line: "{{ ansible_date_time.weekday }} {{ ansible_date_time.month }} {{ ansible_date_time.day }} {{ ansible_date_time.time }} {{ ansible_date_time.tz }} {{ ansible_date_time.year }}, {{ ansible_run_tags }}, {{ ansible_user }}"

If provided in a playbook as very last task it will write a log entry on the Remote Node and if the execution hasn't failed before. By doing this one can have some more centralized information on Remote Node(s) if roles, playbooks, tasks, etc. where applied, when and by whom. Or just adjust the information to be logged with parameter line.

Centralized Log

It is assumed that the log directory (i.e. /var/log/ansible) exits and permissions are given. Furthermore, gather_facts: true and to have the log written on a specific node, it needs to be delegated to it.

- name: "Log report result"
  delegate_to: localhost # or defined Control Node
  lineinfile:
    path: "/var/log/ansible/memory/report.{{ ansible_date_time.year }}{{ ansible_date_time.month }}{{ ansible_date_time.day }}.log"
    create: yes
    line: "{{ ansible_hostname }}, {{ memhigusage.stdout }}"
  when: memsec.stdout | int >= 90 or mem1sec.stdout | int >= 90 or mem2sec.stdout | int >= 90 or mem3sec.stdout | int >= 90

To make it clear from where the information was coming from it will be necessary to log the value together with the name of the Remote Node.

Some Notes

The use case seems not to be creating a log or report but collecting a fact about the system.

Whereby distributed logs may have its justification as shown in the example, for a centralized log there are better approaches available. Specifically for the memory utilization example it is recommended to distribute the whole processing upfront to the Remote Node(s). To do so one may Adding custom facts and implement facts.d or local facts, adjust the script Get-Memory-Utilization.sh and distribute it on the Remote Node(s).

Otherwise and if not doing so one would implement already available functionality within Ansible and playbooks. Furthermore one will create unnecessary code and complexity, higher dependencies and environment utilization. higher maintenance costs, and so on. This all should be avoided in any case.

Are there other ways to get the remote node status or memory consumption?

Sure, if implementing a Dynamic Custom Fact just for memory utilization might not be feasible, one may have a look into already existing solutions like Prometheus Node Exporter. It can easily be deployed according Running node_exporter with Ansible.