Is there a way to make a python aws-lambda on a local machine which can read and write data from an S3 bucket. I can get this to run on a lambda in AWS's web-page with the following code with no problems.
import json
def lambda_handler(event, context):
# TODO implement
now = datetime.datetime.now()
cur_day = "{}{}{}".format(now.strftime('%d'),now.strftime('%m'),now.year)
print(cur_day)
my_contents = get_data_from_s3_file('myBucket', 'myFile')
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}
def get_data_from_s3_file(bucket, my_file):
"""Read the contents of the file as a string and split by lines"""
my_data = s3.get_object(Bucket=bucket, Key=my_file)
my_text = my_data['Body'].read().decode('utf-8').split('\n')
return my_text
The issue is that this is a terrible environment to write and debug code so I would like to do it on my own local machine. I have set up AWS-CLI and installed an app that lets you run lambda code in a local environment called 'python-lambda-local', as shown here.
pip install python-lambda-local
python-lambda-local -l lib/ -f lambda_handler -t 5 pythonLambdaLocalTest.py event.json
The file 'pythonLambdaLocalTest.py' contains the same code that I ran on AWS from the console - the advantage here is that I can use an IDE such as visual studio code to write it. If I run it without calling 'get_data_from_s3_file' then the code seems to run fine on the local machine and 'cur_day' is printed to the console. However, if I run the full script and try to connect to the bucket then I get the following error:
raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the
endpoint URL: "https://myBucket/myfile"
Does anyone have a method to connect to the s3 from the local machine? I'm sure there must be a way to use aws-cli? or the serverless-application-model (sam)? However I can't find any guides which are complete enough to follow.
I've also tried downloading the .yaml file from the console and putting it in the local directory and running:
sam local start-api -t pythonLambdaLocalTest.yaml
and I get:
2019-01-21 16:56:30 Found credentials in shared credentials file: ~/.aws/credentials
Error: Template does not have any APIs connected to Lambda functions
This suggests that potentially an api could connect my local machine to the aws s3 bucket but I have very little experience in setting this kind of thing up and am struggling with the jargon. Any help on getting this approach to run would be great? I've recently started using docker so some approach using this would also be great?
I've also tried this approach here and can see my lambda functions listed in visual studio code but I can't seem to see or edit any of the code and there is no obvious link to do so - most of the support seems to be around node.js and my lambda's are python.
I also realise that cloud9 is an option but appears to require a running EC2 instance which I would rather not pay for.
I have tried a lot of approaches but there doesn't seem to be any complete guides. Any help highly appreciated!
