0

I'm trying to read some json file from s3 bucket and then trying to post the data into rds and redshift(using a post api developed/intended for that).

I approached to do this in 2 ways using boto3. Below are those:

1st way:

import boto3
import json
import requests

url = 'xxxx.us-east-1.elb.amazonaws.com/v1'
headers = {'content-type': 'application/json', 'Host': 'development.my.dns.com'}
endpoint = url+'/my/post/endpoint'
s3_client = boto3.client("s3")
fileObj = s3_client.get_object(Bucket='my-bucket-name', Key='my-key-name'])
data = fileObj['Body'].read().decode('utf-8')
with requests.request('POST', endpoint, data=data, headers=headers, auth=(username, pwd), verify=False, stream=True) as r:
    print("Status Code:",r.status_code)

2nd way:

import boto3
import json
import requests

url = 'xxxx.us-east-1.elb.amazonaws.com/v1'
headers = {'content-type': 'application/json', 'Host': 'development.my.dns.com'}
endpoint = url+'/my/post/endpoint'
s3_res = boto3.resource('s3')
contentObj = s3_res.Object('my-bucket-name', 'my-key-name')
fileContent = contentObj.get()['Body'].read().decode('utf-8')
data = json.dumps(json.loads(fileContent))
with requests.request('POST', endpoint, data=data, headers=headers, auth=(username, pwd), verify=False, stream=True) as r:
    print("Status Code:",r.status_code)

Basically everything is same(url, endpoint, headers, requests.request) and type(data) is <class 'str'> in both the approaches. But the status codes are always different.

The 2nd way gives Status Code: 200. But the 1st way gives Status Code: 413 or Status Code: 502 randomly

Can any one please explain why is it this way? what's different in the above two approaches? I'm trying to understand what's going on when boto3 read() the data differently.

Kingz
  • 1,657
  • 3
  • 18
  • 29
  • Could you please confirm if the `data` variable contains exactly the same thing in both cases, including all field values and data types. – HenryTK Jun 26 '19 at 18:10
  • Potentially relevant answers here: https://stackoverflow.com/questions/42809096/difference-in-boto3-between-resource-client-and-session – Tommy Jun 26 '19 at 19:46
  • Yes the `data` content in both the cases is same. Except in 1st way, the data is indented and in the 2nd way it is not. – Kingz Jun 26 '19 at 19:50

0 Answers0