How to access information from config.json file to a python file?

Question

The problem is I'm unable to access the information from config.json file to my python file I have provided the JSON data and python code bellow I have tried everything in the request module but I can access the response without the config file but, I need with config file The following is a json file

{
    "api_data": {
        "request_url": "https://newapi.zivame.com/api/v1/catalog/list",
        "post_data" : {"category_ids" : "948",
                        "limit" : "10000"},
        "my_headers":{"Content-Type": "application/json"}
    },
    "redshift":{
        "host":"XXX.XXXX.XXX",
        "user":"XXXX",
        "password":"XXXXXXXX",
        "port": 8080,
        "db":"XXXX"
    },
    "s3":{
        "access_key":"XXXXXXXXX",
        "secret_key":"XXXXXXXXXX",
        "region":"XX-XXXXX-1",
        "path":"XXXXXXXXXXXX/XXX",
        "table":"XXXXXX",
        "bucket":"XXXX",
        "file": "XXXXXX",
        "copy_column": "XXX",
        "local_path": "XXXXX"
    },
    "csv_file": {
        "promo_zivame": ""
    }
}

and this is the program

#!/usr/bin/python                                                                                                                                                                 
import json                                                                                                                                                                       
import psycopg2                                                                                                                                                                   
import requests                                                                                                                                                                   
import os

BASE_PATH = os.path.dirname(os.path.realpath(__file__))

with open(BASE_PATH+'/config.json') as json_data_file:                                                                                                                            
    data = json.load(json_data_file)

#api_config = data['api_data']                                                                                                                                                    
#redshift = data['redshift']                                                                                                                                                      
s3_config = data['s3']

#x = print(api_config.get('request_url'))

class ApiResponse:                                                                                                                                                                
    #api response                                                                                                                                                                 
    def api_data(self, api_config):                                                                                                                                               
        print("starting api_data")                                                                                                                                                
        try:                                                                                                                                                                      
            self.ApiResponse = requests.post(api_config['request_url'], api_config['post_data'], api_config['my_headers'])                                                        
            data_1 = self.ApiResponse                                                                                                                                             
            #data = json.dump(self.ApiResponse)                                                                                                                                   
            print("API Result Response")                                                                                                                                          
            print(())                                                                                                                                                             
            print(self.ApiResponse)                                                                                                                                               
            return (self.ApiResponse)

        except Exception:                                                                                                                                                         
            print("response not found")                                                                                                                                           
            return False

    def redshift_connect(self, redshift):                                                                                                                                         
        try:                                                                                                                                                                      
            # Amazon Redshift connect string                                                                                                                                      
            self.con = psycopg2.connect(                                                                                                                                          
                host=redshift['host'],                                                                                                                                            
                user=redshift['user'],                                                                                                                                            
                port=redshift['port'],                                                                                                                                            
                password=redshift['password'],                                                                                                                                    
                dbname=redshift['db'])                                                                                                                                            
            print(self.con)                                                                                                                                                       
            return self.con                                                                                                                                                       
        except Exception:                                                                                                                                                         
            print("Error in Redshift connection")                                                                                                                                 
            return False                                                                                                                                                          
def main():                                                                                                                                                                       
    c1 = ApiResponse()                                                                                                                                                                   
    api_config = data['api_data']                                                                                                                                                 
    redshift = data['redshift']                                                                                                                                                   
    c1.api_data(api_config)                                                                                                                                                       
    c1.api_data(data)                                                                                                                                                             
    c1.redshift_connect(redshift)

if __name__=='__main__':                                                                                                                                                          
    main()

The json loading stuff is fine... but what is "c1 = data()" ? — Mike Guelfi, Jan 13 '20 at 06:25
Is there an error message? What is not working exactly? `data` is blank? Please provide some more information — Emrah Diril, Jan 13 '20 at 07:33
`starting api_data API Result Response False {'Content-Type': 'application/json;charset=UTF-8', 'Date': 'Mon, 13 Jan 2020 07:36:37 GMT', 'Expires': 'Thu, 01 Jan 1970 00:00:00 GMT', 'Server': 'nginx', 'Set-Cookie': 'visited=yes', 'Content-Length': '138', 'Connection': 'keep-alive'} > 415` — Gautham Arcot, Jan 13 '20 at 07:41
b'{"timestamp":1578901648382,"status":415,"error":"Unsupported Media Type","message":"Unsupported Media Type","path":"/api/v1/catalog/list"}' — Gautham Arcot, Jan 13 '20 at 07:48
Are you sure you are using that REST endpoint correctly? `data` seems to be getting loaded correctly and `api_config` has all the information from your json file. It seems like that endpoint doesn't like json data? — Emrah Diril, Jan 13 '20 at 07:53
yes, Sir the rest endpoint is correct i have checked in postman im getting the response for the data im posting with headers — Gautham Arcot, Jan 13 '20 at 07:58
Try `requests.post(api_config['request_url'], json=api_config['post_data'], headers=api_config['my_headers'])`. Note the keywords `json` and `headers`. — , Jan 13 '20 at 08:30
@JustinEzequiel hey thanks i got 200 response, but can you explaine what is the cause — Gautham Arcot, Jan 13 '20 at 08:35
@JustinEzequiel I have tried as you said but I'm getting timeout, because the data is vast any alternatives? — Gautham Arcot, Jan 13 '20 at 08:45

score 0 · Accepted Answer · answered Jan 13 '20 at 08:45

0

Third argument to requests.post() is json. To provide headers, you need to use the name of the argument explicitly as @JustinEzequiel suggested. See the requests doc here: 2.python-requests.org/en/v1.1.0/user/quickstart/#custom-headers

requests.post(api_config['request_url'], json=api_config['post_data'], headers=api_config['my_headers'])

answered Jan 13 '20 at 08:45

Emrah Diril

1,687
1
19
27

+1 yeah sir i have seen it but as the data is vast im getting time, is it okay to store the response json in a vairable and then pass it? – Gautham Arcot Jan 13 '20 at 08:56
Hmmm I'm not sure. Do you mean `api_config['post_data']` is too big and it's causing timeout? You might be better off asking that as a separate new question – Emrah Diril Jan 13 '20 at 09:02
okay sir and also I have one problem with this can you help? like its basic, but im learning like i have so pass the response body to another program i will ask in one more question now please do review and do the needful please – Gautham Arcot Jan 13 '20 at 09:05
im unable to post one more question could you provide me your linked in or something so that i can text you please!! – Gautham Arcot Jan 13 '20 at 09:26

score 0 · Answer 2 · answered Jan 14 '20 at 00:27

Borrowing code from https://stackoverflow.com/a/16696317/5386938

import requests

api_config = {
    "request_url": "https://newapi.zivame.com/api/v1/catalog/list",
    "post_data" : {"category_ids" : "948", "limit" : "10000"},
    "my_headers":{"Content-Type": "application/json"}
    }
local_filename = 'the_response.json'
with requests.post(api_config['request_url'], json=api_config['post_data'], headers=api_config['my_headers'], stream=True) as r:
    r.raise_for_status()
    with open(local_filename, 'wb') as f:
        for chunk in r.iter_content(chunk_size=8192): 
            if chunk: # filter out keep-alive new chunks
                f.write(chunk)

saves the response into a file ('the_response.json') you can then pass around. Note the stream=True passed to requests.post

thank you sir, and @JustinEzequiel do you have any idea how to access variables from a class's function's in try block variable into another file — Gautham Arcot, Jan 14 '20 at 03:56
and the reason im using a json file because i need it to be secure theres a follow up program for this so — Gautham Arcot, Jan 14 '20 at 04:15

How to access information from config.json file to a python file?

2 Answers2