1

I have a working code in Jupyter notebook to transpose a CSV file. Trying similar thing in AWS Lambda and not working.

Here is my source file.

name,address,phone
john,chicago,1111111111
doe,newyork,2222222222

expected output -->

name,john,doe,
address,chicago,newyork
phone,1111111111,2222222222

Here is my working code in Jupyter

import numpy as np
with open('/Users/test/Desktop/python/pythonnew.txt') as file:
    lis = [x.replace('\n', '').split(',') for x in file]

x = np.array(lis)

print(x.T)

Trying similarly in Lambda and it's not working. Here is my lambda python code.

import boto3
import numpy as np

s3 = boto3.client('s3')


def lambda_handler(event, context):
    data = s3.get_object(Bucket='s3dummy', Key='input/input.txt')

    contents = data['Body'].read()

    output = contents.decode('utf-8')

    lis = [x.split(',') for x in output]

    x = np.array(lis)

    print(x.T)

Any help is greatly appreciated. I am a newbie to AWS as well as Python. So, I apologize in advance if this easy and for wasting your time

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
appleios
  • 9
  • 2
  • Are you trying to print out in lambda or rewrite the csv file? I'm a bit confused on that portion – n_estrada Apr 22 '20 at 17:49
  • yes, for now I am just printing to see how the output looks like, eventually I will be writing that output to a new file in S3 bucket. – appleios Apr 22 '20 at 17:50
  • and are you running locally or are you uploading to lambda then running? – n_estrada Apr 22 '20 at 21:36
  • Uploaded the code to lambda and running there. – appleios Apr 22 '20 at 22:00
  • 1
    AWS Lambda does not include the `numpy` library. See: [AWS Lambda with Pandas and NumPy - Ruslan Korniichuk - Medium](https://medium.com/@korniichuk/lambda-with-pandas-fd81aa2ff25e). However, you could perform the transformation purely using Python, thereby avoiding the need for `numpy`. See: [How to transpose a 2D list array using loops in python?](https://stackoverflow.com/a/40442791/174777) – John Rotenstein Apr 23 '20 at 00:02
  • Thanks John. I added the numpy package in layer. Initially i was getting the module not found error. After adding a layer, it was fine. Problem now is output, when i load the data into list it is taking each character of the file and loading into list instead of splitting based on comma. same code is working fine in jupyter. – appleios Apr 23 '20 at 02:10

0 Answers0