JSON error checking using try and except doesn't work

Question

I am calling an API every 5 minutes to download data but there are some occasions when the correct JSON file is not returned. In these cases I want to use the previous dataset as a substitute. I have the following code that runs but I still get missed data.

with open(r'C:\Users\david\GrowattAPI\plantData.json', 'r+') as file:
    try:
        shutil.copyfile(r'C:\Users\david\GrowattAPI\plantData.json',
                        r'C:\Users\david\GrowattAPI\plantData_old.json')
        data = json.load(file)
        file.close()
    except:
        shutil.copyfile(r'C:\Users\david\GrowattAPI\plantData.json',
                        r'C:\Users\david\GrowattAPI\plantData_error.json')
        file.close()
        f = open(r'C:\Users\david\GrowattAPI\plantData_old.json')
        data = json.load(f)
        f.close()

There has never been any error file written to indicate missed data but there are gaps. I am thinking that the except code never executes. I have seen JSON file return with just {}.

I have tried json.decoder.JSONDecodeError, ValueError, and IndexError individually and as a combination.

Is there a way to capture all exceptions and just use the previous dataset?

Update

I have done some more investigating by capturing 24 hours of 5 minute data. From 288 returns, 7 of the JSON files had issues, 4 of the them were {} and the balance had a subset of standard JSON file that is returned. This is a consistent problem every day. I only want 6 data points out of the JSON file that has 335 lines in it. The data I pick up is nested to 5 levels. I reference it as follows

SolarGeneration = data['332761']['devices']['NTCIA13017']['statusData']['ppv']
Voltage = data['332761']['devices']['NTCIA13017']['statusData']['vac1']
BatteryDischarge = data['332761']['devices']['NTCIA13017']['statusData']['pdisCharge1']
BatteryCharge = data['332761']['devices']['NTCIA13017']['statusData']['chargePower']
Consumption = data['332761']['devices']['NTCIA13017']['statusData']['pLocalLoad']
SOC = data['332761']['devices']['NTCIA13017']['statusData']['SOC']

I am now trying to use jsonschema to see if the tags of the 6 data points I want are contained in the JSON file that is returned.

import json
from jsonschema import validate

def validateJson(data):
 try:
    validate(instance=data, schema=good_schema)
 except jsonschema.exceptions.ValidationError as err:
    return False
 return True

# schema to check against
good_schema = {
 "type": "object",
 "properties": {
     "vac1": {"type": "number"},
     "SOC": {"type": "number"},
     "pdisCharge1": {"type": "number"},
     "ppv": {"type": "number"},
     "pLocalLoad": {"type": "number"},
     "chargePower": {"type": "number"},
 },
}

f = open(r'C:\Users\david\PycharmProjects\JSON_TEST\2110.json')
data = json.load(f)
isValid = validateJson(data)
if isValid:
    print(data)
    print("Given JSON data is Valid")
else:
    print(data)
    print("Given JSON data is InValid")

Even when I load an empty file {} the code returns a valid TRUE.

Any ideas?

The only way `data` should end up as an empty dict is if the file literally contained `{}`. (Unless it is initialized to an empty dict beforehand, which you did not show) — John Gordon, Mar 21 '22 at 23:50
When you use `with` to open a file you don't need to worry about closing it as that will happen automatically even if an exception occurs. That said, I **strongly** suggest avoiding "bare" `except` clauses because they can hide things you never thought of going wrong. Change it to `except Exception as exc:` then `print(exc)` or log it when one occurs so you know what's going wrong. — martineau, Mar 21 '22 at 23:51
Does this answer your question? [How can I write a \`try\`/\`except\` block that catches all exceptions?](https://stackoverflow.com/questions/4990718/how-can-i-write-a-try-except-block-that-catches-all-exceptions) — tevemadar, Mar 22 '22 at 12:23

score 0 · Answer 1 · answered Mar 21 '22 at 23:20

0

You can use the Exception class as an argument to your except statement.

There's a similar solution here:

How can I write a `try`/`except` block that catches all exceptions?

answered Mar 21 '22 at 23:20

Chase Jones

121
6

JSON error checking using try and except doesn't work

1 Answers1