0

I'm making an API call and I'm getting an None value as I loop through my file. I would place my csv in here, but it's over 100k records.

My code:

import csv
import xml.etree.ElementTree as ET
import xml.dom.minidom  
import requests
import json
import pandas as pd
from pandas.io.json import json_normalize

#Storing results
api_results = []
error_results = []
none_results = []


print('Starting XML loop')

with open("C:/Users/template_v2.csv") as f: 
   reader = csv.DictReader(f)

   # WRITING TO XML NODES 
   for i, row in enumerate(reader, start=1):
       # INITIALIZING XML FILE
       root = ET.Element('cbcalc')

       icdNode = ET.SubElement(root, "icdcodes")
        #handling first level ICD codes
       for code in row['D'].split('~'):
           ET.SubElement(icdNode, "code").text = code

           #handling Client ID, state, country, age, job class and output
       ET.SubElement(root, "clientid").text = row['CLAIM_NUM']
       ET.SubElement(root, "state").text = row['BEN_ST']
       ET.SubElement(root, "country").text = "US"  
       ET.SubElement(root, "age").text = row['AGE']
       ET.SubElement(root, "jobclass").text = "1" 
       ET.SubElement(root, "fulloutput").text ="Y"

       #handling the cfactors:
       cfNode = ET.SubElement(root, "cfactors")
       for k in ['legalrep', 'depression', 'diabetes',
                 'hypertension', 'obesity', 'smoker', 'subabuse']:
           ET.SubElement(cfNode, k.lower()).text = str(row[k])

       psNode = ET.SubElement(root, "prosummary")

       psicdNode = ET.SubElement(psNode, "icd")
       for code in row['P'].split('~'):
           ET.SubElement(psNode, "code").text = code

       psndcNode = ET.SubElement(psNode, "ndc")
       for code in row['NDC_codes'].split('~'):
           ET.SubElement(psNode, "code").text = code 

       cptNode = ET.SubElement(psNode, "cpt")
       for code in row['CPT_codes'].split('~'):
           ET.SubElement(cptNode, "code").text = code

       ET.SubElement(psNode, "hcpcs")

       doc = ET.tostring(root, method='xml', encoding="UTF-8")
       response = requests.post(target_url, data=doc, headers=login_details)
       response_data = json.loads(response.text)
       if type(response_data)==dict and 'error' in response_data.keys():
           error_results.append(response_data)
       elif response_data == None or response_data == '':
           none_results.append(response_data)
       else:
           api_results.append(response_data)

print('creating dataframe')
strategic_df = pd.json_normalize(api_results)

print("Writing out csv file")
strategic_df.to_csv(r'C:\Users\_dataframe2.csv', index = False, header=True)

Here is my error message:

Traceback (most recent call last):
  File "c:\Users\Python\sc_ras_api.py", line 66, in <module>
    response_data = json.loads(response.text)
  File "C:\Users\AppData\Local\Programs\Python\Python38-32\lib\json\__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "C:\Users\AppData\Local\Programs\Python\Python38-32\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\AppData\Local\Programs\Python\Python38-32\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

What did I break and how do I fix it?

BloodKid01
  • 111
  • 14
  • The first suspicion is that your `response.text` doesn't actually have a JSON document in it. – Charles Duffy Jan 06 '21 at 16:23
  • Log the actual response content. Doing so will also let you reduce your question towards being a [mre] -- once you know actual values, you can eliminate code involved in generating those values from your question (by hardcoding the actual response or a subset with similar behavior should it be correct/as-intended), _or_ focus in on that code and generate an example that reproduces the problem it has while eliminating everything that isn't strictly necessary to make that example work. – Charles Duffy Jan 06 '21 at 16:24
  • @CharlesDuffy when I run this code with 2000 records I typically have a JSON document in it. Nevertheless, how do I account for that? – BloodKid01 Jan 06 '21 at 16:26
  • What "that" is it you're asking how to account for? We need to know the failure mode before we can tell you how to account for it -- so **add the logging** so we can find out what the failure mode is. – Charles Duffy Jan 06 '21 at 16:27
  • @CharlesDuffy how do I log the response content? – BloodKid01 Jan 06 '21 at 16:27
  • However you like. `print(repr(response.text))` is one very low-rent way to get there. Can also use the `logging` module from the Python standard library if you prefer. – Charles Duffy Jan 06 '21 at 16:27
  • Right now this code can't be run by anyone but you -- nobody else has the data files, nobody has the URLs to post them to -- so nobody but you can evaluate whether a proposed fix has actually successfully solved the problem. Part of the goal of us insisting on [mre]s is so we can make sure that (1) our answers actually solve the problem; and perhaps even more importantly (2) the question/answer pairs are something someone else with a similar issue can effectively find and learn from. – Charles Duffy Jan 06 '21 at 16:31
  • ...if it turns out `response.text` has content that isn't what you expect, then inspecting other parts of the `response` object may be the place to go after that. If you're sometimes getting 3xx or 5xx errors from the server, then that's something you'd want to write code to handle, after trying to figure out _why_ (rate limiting? server-side bug? something else?). – Charles Duffy Jan 06 '21 at 16:32
  • @CharlesDuffy fair enough. how would I use the logging module to get the error message needed to get further assistance on this issue? – BloodKid01 Jan 06 '21 at 16:35
  • [log all requests from the Python requests module](https://stackoverflow.com/questions/16337511/log-all-requests-from-the-python-requests-module) is one place you might start. – Charles Duffy Jan 06 '21 at 16:53

0 Answers0