I am creating a lambda using Python to create a file in s3 bucket but its only creating one row. Need iteration based creation

Question

I am creating an AWS Lambda function using Python to create a file in an S3 bucket but it is only creating one row. Need iteration based creation.

Below is my code:

import json
import boto3

s3 = boto3.client('s3')

def lambda_handler(event, context):
    bucket ='xyz'

    eventToUpload = {}
    eventToUpload['ITEM_ID'] = 1234
    eventToUpload['Name'] = John
    eventToUpload['Office'] = NY

    fileName = 'testevent' + '.json'

    uploadByteStream = bytes(json.dumps(eventToUpload).encode('UTF-8'))

    s3.put_object(Bucket=bucket, Key=fileName, Body=uploadByteStream)

    print('Put Complete')

Above code is creating a json file in the S3 bucket but I want to iteration of say 5 so that file has 5 record.

Your code only seems to be creating one `eventToUpload`. Where would you obtain the information to output multiple records? — John Rotenstein, Dec 15 '21 at 10:39
condition 1 ) same value for multiple records and 2) fetching the ghz file from s3 and appending the records — Farhan Uddin Quidwai, Dec 15 '21 at 10:59

score 0 · Answer 1 · edited May 10 '23 at 20:34

AWS S3 doesn't by default, nor does it support, append content to an existing file.

You can either:

Loop through your events on your Python script to create multiple files with an uniquely identified fileName prefix/suffix, say, ITEM_ID;
Read and parse the existing file beforehand, if any, then append your events to it and write/replace the file.

This question seems to have useful answers to your case as well.

I am creating a lambda using Python to create a file in s3 bucket but its only creating one row. Need iteration based creation

1 Answers1