JSON load function gives Extra Data value error

Question

When I read JSON file it gives me following error:

json.decoder.JSONDecodeError: Extra data: line 16043 column 2 (char 370886)

This error seems to be valid as json.load does not decode multiple json objects and my JSON contains 2 or more objects.

{ "log": [ 
 {
   "code": "info",
   "message": {"text": "[info] Activation of plug-in abcd rule processor (xule) successful, version Check version using Tools->Xule->Version on the GUI or --xule-version on the command line. - xule "},
   "refs": [{"href": "xule"}],
   "level": "info"
 }
,
{
   "code": "xyz.F1.all.7",
   "level": "error",
   "message": {
                "text": "[xyz.F1.all.7] The value for ForResale with a value of 63 has a unit of utr:MWh. This concept allows units of utr:MWh.\n\nElement : xyz:ForResale\nPeriod : 2016-01-01 to 2016-12-31\nUnit : utr:MWh\n\nRule Id:xyz.F1.all.7 - TestUtilitiesInc-428-2016Q4F1.abcd 4114",
                "severity": "error",
                "cid": "63096080",
                "filing_url": "C:\\Users\\TEST\\Desktop\\TestUtilitiesInc-428-2016Q4F1.abcd"
             },
   "refs": [{"href": "xule"}]      

}]
}
{ "log": [ 
 {
   "code": "info",
   "message": {"text": "[info] Activation of plug-in abcd rule processor (xule) successful, version Check version using Tools->Xule->Version on the GUI or --xule-version on the command line. - xule "},
   "refs": [{"href": "xule"}],
   "level": "info"
 }
,
{
   "code": "xyz.F1.all.7",
   "level": "error",
   "message": {
                "text": "[xyz.F1.all.7] The value for ForResale with a value of 63 has a unit of utr:MWh. This concept allows units of utr:MWh.\n\nElement : xyz:ForResale\nPeriod : 2016-01-01 to 2016-12-31\nUnit : utr:MWh\n\nRule Id:xyz.F1.all.7 - TestUtilitiesInc-428-2016Q4F1.abcd 4114",
                "severity": "error",
                "cid": "63096080",
                "filing_url": "C:\\Users\\TEST\\Desktop\\TestUtilitiesInc-428-2016Q4F1.abcd"
             },
   "refs": [{"href": "xule"}]


}]}

I came across this article for resolution: Python json.loads shows ValueError: Extra data which says I need to add my log dictionaries in json.dumps() prior to json.loads(). But, I am unable to get it done. Please can anyone help me.

This is my python code:

with open('C:/Users/Desktop/SampleTestFiles/logfile.json', encoding="utf-8") as json_file:
    data = json.load(json_file)

E.Serra · Accepted Answer · 2019-11-27T15:21:40.807

according to jsonlint:

Error: Parse error on line 27:
...le"          }]      }   ]} {    "log": [{           "cod
------------------^
Expecting 'EOF', '}', ',', ']', got '{'

In other words , you have 2 jsons in one, should be a list, as in

[{
        "log": [{
                "code": "info",
                "message": {
                    "text": "[info] Activation of plug-in abcd rule processor (xule) successful, version Check version using Tools->Xule->Version on the GUI or --xule-version on the command line. - xule "
                },
                "refs": [{
                    "href": "xule"
                }],
                "level": "info"
            },
            {
                "code": "xyz.F1.all.7",
                "level": "error",
                "message": {
                    "text": "[xyz.F1.all.7] The value for ForResale with a value of 63 has a unit of utr:MWh. This concept allows units of utr:MWh.\n\nElement : xyz:ForResale\nPeriod : 2016-01-01 to 2016-12-31\nUnit : utr:MWh\n\nRule Id:xyz.F1.all.7 - TestUtilitiesInc-428-2016Q4F1.abcd 4114",
                    "severity": "error",
                    "cid": "63096080",
                    "filing_url": "C:\\Users\\TEST\\Desktop\\TestUtilitiesInc-428-2016Q4F1.abcd"
                },
                "refs": [{
                    "href": "xule"
                }]

            }
        ]
    },
    {
        "log": [{
                "code": "info",
                "message": {
                    "text": "[info] Activation of plug-in abcd rule processor (xule) successful, version Check version using Tools->Xule->Version on the GUI or --xule-version on the command line. - xule "
                },
                "refs": [{
                    "href": "xule"
                }],
                "level": "info"
            },
            {
                "code": "xyz.F1.all.7",
                "level": "error",
                "message": {
                    "text": "[xyz.F1.all.7] The value for ForResale with a value of 63 has a unit of utr:MWh. This concept allows units of utr:MWh.\n\nElement : xyz:ForResale\nPeriod : 2016-01-01 to 2016-12-31\nUnit : utr:MWh\n\nRule Id:xyz.F1.all.7 - TestUtilitiesInc-428-2016Q4F1.abcd 4114",
                    "severity": "error",
                    "cid": "63096080",
                    "filing_url": "C:\\Users\\TEST\\Desktop\\TestUtilitiesInc-428-2016Q4F1.abcd"
                },
                "refs": [{
                    "href": "xule"
                }]


            }
        ]
    }
]

TRY SOMETHING LikE:


import json

mylist = []
with open('C:/Users/Desktop/SampleTestFiles/logfile.json', encoding="utf-8") as f:
    new_el = ''
    for l in f:
       new_el += l.rstrip('\n')
       try:
          sub = json.loads(new_el)
          mylist.append(sub)
          new_el = ''
       except:# json.decoder.JSONDecodeError:
          pass
print(mylist)

This prints: [{u'log': [{u'message': {u'text': u'[info] Activation of plug-in abcd rule processor (xule) successful, version Check version using Tools->Xule->Version on the GUI or --xule-version on the command line. - xule '}, u'code': u'info', u'refs': [{u'href': u'xule'}], u'level': u'info'}, {u'message': {u'text': u'[xyz.F1.all.7] The value for ForResale with a value of 63 has a unit of utr:MWh. This concept allows units of utr:MWh.\n\nElement : xyz:ForResale\nPeriod : 2016-01-01 to 2016-12-31\nUnit : utr:MWh\n\nRule Id:xyz.F1.all.7 - TestUtilitiesInc-428-2016Q4F1.abcd 4114', u'filing_url': u'C:\Users\TEST\Desktop\TestUtilitiesInc-428-2016Q4F1.abcd', u'severity': u'error', u'cid': u'63096080'}, u'code': u'xyz.F1.all.7', u'refs': [{u'href': u'xule'}], u'level': u'error'}]}, {u'log': [{u'message': {u'text': u'[info] Activation of plug-in abcd rule processor (xule) successful, version Check version using Tools->Xule->Version on the GUI or --xule-version on the command line. - xule '}, u'code': u'info', u'refs': [{u'href': u'xule'}], u'level': u'info'}, {u'message': {u'text': u'[xyz.F1.all.7] The value for ForResale with a value of 63 has a unit of utr:MWh. This concept allows units of utr:MWh.\n\nElement : xyz:ForResale\nPeriod : 2016-01-01 to 2016-12-31\nUnit : utr:MWh\n\nRule Id:xyz.F1.all.7 - TestUtilitiesInc-428-2016Q4F1.abcd 4114', u'filing_url': u'C:\Users\TEST\Desktop\TestUtilitiesInc-428-2016Q4F1.abcd', u'severity': u'error', u'cid': u'63096080'}, u'code': u'xyz.F1.all.7', u'refs': [{u'href': u'xule'}], u'level': u'error'}]}]

For testing purpose I gave only two logs. But my actual file has 60K logs for which I need to parse the file using python. — user2961127, Nov 27 '19 at 14:53
well you need to make them into a list, try turning them into a dict somehow, but your problem is pbbly you just printed them and are trying to load them as if they were a json and they are not a valid json, just a bunch of jsons dumped in the same file — E.Serra, Nov 27 '19 at 15:02

JSON load function gives Extra Data value error

1 Answers1