2

Trying to use this Ansible community.aws.aws_ssm module but keep getting "ERROR! A worker was found in a dead state"

Could not find any solution for this case, only outdated issues with Ansible throwing out of memory errors.

Playbook

- name: Install a Nginx Package
  hosts: all
  vars:

    profile: default
    ansible_connection: aws_ssm
    ansible_aws_ssm_bucket_name: fleetup.logs
    ansible_aws_ssm_region: us-west-2

  tasks:
    - name: Install package
      raw: yum install -y awscli

Below is output from cli.

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************************
task path: /Users/rerser/Documents/VSCode_Worspace/Hillel_devops/ansible/test_ssm.yml:1
 43452 1641248909.66836: sending task start callback
 43452 1641248909.66837: entering _queue_task() for i-0284369df994a8ec5/gather_facts
 43452 1641248909.66838: Creating lock for gather_facts
 43452 1641248909.67034: worker is 1 (out of 1 available)
 43452 1641248909.67079: exiting _queue_task() for i-0284369df994a8ec5/gather_facts
 43452 1641248909.67146: done queuing things up, now waiting for results queue to drain
 43452 1641248909.67148: waiting for pending results...
 43461 1641248909.67400: running TaskExecutor() for i-0284369df994a8ec5/TASK: Gathering Facts
 43461 1641248909.67457: in run() - task c25454ab-7f62-17b5-f68b-00000000000c
 43461 1641248909.67472: variable 'ansible_search_path' from source: unknown
 43461 1641248909.67517: calling self._execute()
 43461 1641248909.67550: variable 'ansible_connection' from source: play vars
 43461 1641248909.67557: variable 'omit' from source: magic vars
 43461 1641248909.67646: variable 'omit' from source: magic vars
 43461 1641248909.67662: variable 'omit' from source: magic vars
 43461 1641248909.67681: variable 'ansible_connection' from source: play vars
 43461 1641248909.67683: variable 'ansible_connection' from source: play vars
 43461 1641248909.67707: trying /Users/rerser/Library/Python/3.8/lib/python/site-packages/ansible/plugins/connection
redirecting (type: connection) ansible.builtin.aws_ssm to community.aws.aws_ssm
 43461 1641248909.70869: Loaded config def from plugin (connection/ansible_collections.community.aws.plugins.connection.aws_ssm)
 43461 1641248909.70879: Loading Connection 'ansible_collections.community.aws.plugins.connection.aws_ssm' from /Users/rerser/.ansible/collections/ansible_collections/community/aws/plugins/connection/aws_ssm.py
 43461 1641248909.70900: trying /Users/rerser/Library/Python/3.8/lib/python/site-packages/ansible/plugins/shell
 43461 1641248909.70939: Loading ShellModule 'sh' from /Users/rerser/Library/Python/3.8/lib/python/site-packages/ansible/plugins/shell/sh.py (found_in_cache=True, class_only=False)
 43461 1641248909.70993: Loading ShellModule 'sh' from /Users/rerser/Library/Python/3.8/lib/python/site-packages/ansible/plugins/shell/sh.py (found_in_cache=True, class_only=False)
 43461 1641248909.71013: variable 'ansible_aws_ssm_region' from source: play vars
 43461 1641248909.71016: variable 'ansible_aws_ssm_bucket_name' from source: play vars
 43461 1641248909.71103: Loading ActionModule 'gather_facts' from /Users/rerser/Library/Python/3.8/lib/python/site-packages/ansible/plugins/action/gather_facts.py (found_in_cache=True, class_only=False)
 43461 1641248909.71126: variable 'omit' from source: magic vars
 43461 1641248909.71132: starting attempt loop
 43461 1641248909.71133: running the handler
 43461 1641248909.71149: variable 'ansible_facts' from source: unknown
 43461 1641248909.71347: trying /Users/rerser/Library/Python/3.8/lib/python/site-packages/ansible/modules
 43461 1641248909.71490: _low_level_execute_command(): starting
 43461 1641248909.71502: _low_level_execute_command(): executing: echo ~ubuntu
<i-0284369df994a8ec5> ESTABLISH SSM CONNECTION TO: i-0284369df994a8ec5
 43452 1641248909.90803: RUNNING CLEANUP
ERROR! A worker was found in a dead state
(venv) rerser@Sergiis-MacBook-Pro ansible %
Sergey
  • 43
  • 1
  • 6
  • I am not super convinced that `connection: aws_ssm` and `gather_facts:` are friends; might I suggest trying the "hello, world" of ansible by `{ hosts: all, connection: community.aws.aws_ssm, gather_facts: no, tasks: [ {raw: uptime } ] }` and see how that shakes out? I believe your bare `profile: default` var is also going to be ignored by aws-y plugins; perhaps you wanted `AWS_PROFILE=default` in your `ansible-playbook`'s environment? – mdaniel Jan 03 '22 at 23:14
  • Separately, my sincere thanks for your inclusion of `ANSIBLE_DEBUG=1` output in your question -- that was really helpful! – mdaniel Jan 03 '22 at 23:15
  • Hi, @mdaniel thx for reaching out and showing the correct direction) – Sergey Jan 03 '22 at 23:30
  • I had a similar issue - the solution here worked for (I'm on OSX): https://stackoverflow.com/questions/64586583/how-to-lookup-an-amazon-aws-aws-secret-in-ansible-playbook – jsjw Jan 31 '22 at 08:16

1 Answers1

2

Seems like I have found temporary solution and will try to find the route cause(must be AWS creds/profile is messed up)

For now with no_proxy="*" flag - all worked.

    (venv) rerser@Sergiis-MacBook-Pro ansible % no_proxy="*" ansible-playbook -vvv test_ssm.yml    
    ansible-playbook [core 2.12.1]
      config file = /Users/rerser/Documents/VSCode_Worspace/Hillel_devops/ansible/ansible.cfg
      configured module search path = ['/Users/rerser/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
      ansible python module location = /Users/rerser/Library/Python/3.8/lib/python/site-packages/ansible
      ansible collection location = /Users/rerser/.ansible/collections:/usr/share/ansible/collections
      executable location = /Users/rerser/Library/Python/3.8/bin/ansible-playbook
      python version = 3.8.9 (default, Oct 26 2021, 07:25:53) [Clang 13.0.0 (clang-1300.0.29.30)]
      jinja version = 3.0.3
      libyaml = False
    Using /Users/rerser/Documents/VSCode_Worspace/Hillel_devops/ansible/ansible.cfg as config file
    redirecting (type: inventory) ansible.builtin.virtualbox to community.general.virtualbox
    redirecting (type: inventory) ansible.builtin.aws_ec2 to amazon.aws.aws_ec2
    host_list declined parsing /Users/rerser/Documents/VSCode_Worspace/Hillel_devops/ansible/aws_ec2.yml as it did not pass its verify_file() method
    ansible_collections.community.general.plugins.inventory.virtualbox declined parsing /Users/rerser/Documents/VSCode_Worspace/Hillel_devops/ansible/aws_ec2.yml as it did not pass its verify_file() method
    Parsed /Users/rerser/Documents/VSCode_Worspace/Hillel_devops/ansible/aws_ec2.yml inventory source with ansible_collections.amazon.aws.plugins.inventory.aws_ec2 plugin
    Skipping callback 'default', as we already have a stdout callback.
    Skipping callback 'minimal', as we already have a stdout callback.
    Skipping callback 'oneline', as we already have a stdout callback.

    PLAYBOOK: test_ssm.yml *****************************************************************************************************************************************************************************************************
    1 plays in test_ssm.yml

    PLAY [test] ****************************************************************************************************************************************************************************************************************
    META: ran handlers

    TASK [raw] *****************************************************************************************************************************************************************************************************************
    task path: /Users/rerser/Documents/VSCode_Worspace/Hillel_devops/ansible/test_ssm.yml:22
    redirecting (type: connection) ansible.builtin.aws_ssm to community.aws.aws_ssm
    <i-061b56922bf0984ea> ESTABLISH SSM CONNECTION TO: i-061b56922bf0984ea
    <i-061b56922bf0984ea> SSM CONNECTION ID: sergey.moroz-0ee4cc3f505e9ba1f
    <i-061b56922bf0984ea> EXEC uptime
    <i-061b56922bf0984ea> (0, ' 23:21:43 up 6 days, 22:38,  0 users,  load average: 0.01, 0.00, 0.00\r\r', '')
    <i-061b56922bf0984ea> CLOSING SSM CONNECTION TO: i-061b56922bf0984ea
    changed: [i-061b56922bf0984ea] => {
        "changed": true,
        "rc": 0,
        "stderr": "",
        "stderr_lines": [],
        "stdout": " 23:21:43 up 6 days, 22:38,  0 users,  load average: 0.01, 0.00, 0.00\r\r",
        "stdout_lines": [
            " 23:21:43 up 6 days, 22:38,  0 users,  load average: 0.01, 0.00, 0.00",
            ""
        ]
    }
    META: ran handlers
    META: ran handlers

    PLAY RECAP *****************************************************************************************************************************************************************************************************************
    i-061b56922bf0984ea        : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Playbook:

- name: test
  hosts: all 
  vars:
    aws_profile: default
    ansible_aws_ssm_region: us-west-2
  connection: aws_ssm
  gather_facts: no
  tasks: 
    - raw: uptime
Sergey
  • 43
  • 1
  • 6