Append all logs to text file and send it to slack via AWS lambda python script

Question

I have a python script that collect the data from aws cloudwatch and send it to slack. Its working completely fine. Its sending logs to slack in the form of text in slack channel.

I would want the same logs data to be stored in a text file and send it to slack. I am not into python so, please excuse my code.

In the code below I am trying to create a file and write the txt file with the message variable to store text.

Also remove json=message with files=file_dict

#  json=message
response = requests.post(slack_webhook_url,files=file_dict)

Python

import boto3
import requests
import json

def lambda_handler(event, context):

    print(event)

    build_id = event['detail']['build-id'].split(':')[-1]

    logs_client = boto3.client('logs')

    log_group_name = "/aws/codebuild/" + event['detail']['project-name']
    log_stream_name = build_id

    response = logs_client.get_log_events(
        logGroupName=log_group_name,
        logStreamName=log_stream_name,
        startFromHead=True
    )
    

    message = {
        'text': 'Logs for build {0}:\n'.format(build_id)
    }

    for event in response['events']:
        message['text'] += event['message'] + '\n'

    slack_webhook_url = 'https://hooks.slack.com/services/123123123/123123123/123123123';
    
    
    with open("/tmp/codebuild-logs.txt", "w") as a_file:
        a_file.write(message)
        file_dict = {"/tmp/codebuild-logs.txt": a_file}
        #  json=message
        response = requests.post(slack_webhook_url,files=file_dict)
        if response.status_code == 200:
            return {
                'statusCode': 200,
                'body': 'Message is sent to Slack successfully.'
            }
        else:
            return {
                'statusCode': response.status_code,
                'body': 'Failed to send message to Slack.'
            }

After running this script I get lambda error.

Response
{
  "errorMessage": "write() argument must be str, not dict",
  "errorType": "TypeError",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 36, in lambda_handler\n    a_file.write(message)\n"
  ]
}

Any help will be greatly appreciated. Thanks in advance.

there's no default serialization so that a `dict` can be written to a file. Actually looking at `https://stackoverflow.com/questions/22567306/how-to-upload-file-with-python-requests` , it doesn't look like you want to pass `reqests.post` a file handle, just a dictionary with filename keys and content values. might be able to handle a few different things though — erik258, Apr 05 '23 at 15:36
dictionaries / lists can be turned into json to be serialized into a file in a standard way that will be accessible to any language and many tools. I don't recommend pickling as an alternative. — erik258, Apr 05 '23 at 15:41
Can you correct my code. I am not able to run again. ```files = {'upload_file': open('file.txt','rb')} values={'message': message} response = requests.post(slack_webhook_url, files=files, data=values)``` it says file.txt not found — Santosh, Apr 05 '23 at 16:39

score 0 · Answer 1 · answered Apr 05 '23 at 21:04

This is an example of how to post file content with requests:

import requests

resp = requests.post(
    "https://httpbin.org/post",
    files={"codebuild-logs.txt": "this\nis\nfile\ncontent" },
)
print(resp.json())

Note here I am not opening any file. A reqest.post's files parameter can be a dictionary with filename keys and file content values. (requests does support passing open files, but that doesn't seem valuable to you here).

For you, message is the content you want to send. You need to turn it into a string - for that, json is great.

Here's some example data in a post as the content of a file:

import requests, json

message = [ "example", { "structured": True } ]

resp = requests.post(
    "https://httpbin.org/post",
    files={"codebuild-logs.txt": json.dumps(message) }
)
print(resp.json())

If you run that, you'll get something like:

{'args': {}, 'data': '', 'files': {'codebuild-logs.txt': '["example", {"structured": true}]'}, 'form': {}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Content-Length': '201', 'Content-Type': 'multipart/form-data; boundary=547bda61944fabd988686b4642024c26', 'Host': 'httpbin.org', 'User-Agent': 'python-requests/2.26.0', 'X-Amzn-Trace-Id': 'Root=1-642db855-153edb2f6bd09c9d1b5bd604'}, 'json': None, 'origin': 'xxx.xxx.xxx.xxx', 'url': 'https://httpbin.org/post'}

For me it return failed to send message to slack. Error code 400. not sure how to debug. In cloudwatch it just state error 400. Also need to add 'tmp/codebuild-logs.txt' otherwise its not writable. — Santosh, Apr 06 '23 at 05:08

Append all logs to text file and send it to slack via AWS lambda python script

1 Answers1