0

Following the blog post for enabling running a task on every ecs host, I have created an Ansible role which successfully creates a Cloudwatch event rule which triggers the Lambda function.

The event rule looks ok in the AWS console and it seems to be triggered in the metrics, but the lambda function is not running (The ecs service is not changed).

When simply editing the rule and saving it with no changes the rule starts to work and the ecs service is changed as expected.

This is my original playbook and roles, I can create a simpler example if needed.

Playbook: 

- name: "create lambda and cloudwatch event rules"
  hosts: localhost
  roles:
    - {
        role: aws/lambda/lookup,
        lambda: ecs-task-on-all-hosts,
        lambda_lookup_register_as: lambda_lookup
      }
    - {
        role: aws/cloudwatch/event/rule/create,
        event_rule: ecs-task-on-all-hosts,
        cluster: "{{ cluster }}",
        lambda_ecs_task_on_all_hosts_arn: "{{ lambda_lookup.arn }}"
      }

Role aws/lambda/lookup:

- name: "lookup lambda {{ lambda }}"
  lambda_facts:
    region: "{{ region }}"
    query: config #right now everything we need is given here
    function_name: "{{ lambda }}"
  register: _lambda_function_details

- name: "set lambda lookup result facts"
  set_fact:
    "{{ lambda_lookup_register_as }}":
      arn: "{{ _lambda_function_details.ansible_facts.lambda_facts.function[lambda].function_arn }}"
      name: "{{ _lambda_function_details.ansible_facts.lambda_facts.function[lambda].function_name }}"

Role aws/cloudwatch/event/rule/create: tasks/main.yml

- name: "include variables in file {{ event_rule }}.yml"
  include_vars: "files/{{ event_rule }}.yml"

- name: "verify mandatory parameters were provided"
  include_role:
    name: utilities/verify-parameters
  vars:
    mandatory: "{{ event_mandatory_parameters }}"
  when: event_mandatory_parameters is defined

- name: "create cloudwatch event rule"
  cloudwatchevent_rule:
    region: "{{ region }}"
    name: "{{ event_definition.name }}"
    description: "{{ event_definition.description }}"
    event_pattern: "{{ event_definition.event_pattern | to_json }}"
    targets: "{{ event_definition.targets }}"

File that is being used files/ecs-task-on-all-hosts:

event_mandatory_parameters: [ "cluster", "lambda_ecs_task_on_all_hosts_arn" ]
event_definition:
  name: ecs-task-on-all-hosts
  description: Ensure a task is running on all hosts in the cluster
  event_pattern: |-
    {
      "source": [
        "aws.ecs"
      ],
      "detail-type": [
        "ECS Container Instance State Change"
      ],
      "detail": {
        "clusterArn": [
          "arn:aws:ecs:{{ region }}:{{ account_id }}:cluster/{{ cluster }}"
        ]
      }
    }
  targets:
    - id: lambda_ecs_task_on_all_hosts
      arn: "{{ lambda_ecs_task_on_all_hosts_arn }}"

A similar question was raised here but it is a little bit different, and still unanswered, so I'm providing all the details I can here. Please let me know if more information is needed.

apines
  • 1,254
  • 14
  • 36

1 Answers1

0

You need to add a Lambda function policy allowing CloudWatch Events to invoke your Lambda function. The reason editing the CloudWatch Event makes it work is that it adds the Lambda function policy in the background.

You can check a function's policy with aws lambda get-policy. Before editing the event, you should see no policy, and after editing, you will see a policy allowing CloudWatch Events to invoke the function.

You can use Ansible module lambda_policy to set the policy within Ansible, e.g.

- name: allow CloudWatch to invoke the Lambda function
  lambda_policy:
    region: "{{ aws_region }}"
    function_name: ecs-task-on-all-hosts
    state: present
    statement_id: lambda-cloudwatch-event-rule
    action: lambda:InvokeFunction
    principal: events.amazonaws.com
    source_arn: "{{ event.rule.arn }}"

where event is the return value of cloudwatchevent_rule.

Sources:

David Kretch
  • 390
  • 1
  • 7