0
with open(filepath) as filedata:
    decks = json.load(filedata)

I'm using the above code to try and load a JSON file created by a Scrapy Spider. I recently reran the scraper to get an updated JSON file and the format should be the same but the lines above through the following error when I try to load the new file. (note this isn't appended to any existing file).

    Traceback (most recent call last):
  File "/Users/tonysitu/Documents/workspace/Mtg/ArchetypeAnalysis/analyze/deck_stats.py", line 215, in launcher
    standard = constructFormat()
  File "/Users/tonysitu/Documents/workspace/Mtg/ArchetypeAnalysis/analyze/deck_stats.py", line 98, in constructFormat
    decks = json.load(deckData)
  File "/Users/tonysitu/anaconda3/lib/python3.4/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/Users/tonysitu/anaconda3/lib/python3.4/json/__init__.py", line 318, in loads
    return _default_decoder.decode(s)
  File "/Users/tonysitu/anaconda3/lib/python3.4/json/decoder.py", line 346, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 120 column 835 - line 306 column 945 (char 99352 - 254693)

I've validated the JSON file using various applications on the web and none have raised an error. In addition all the dictionaries are enclosed by a single set of [ ] brackets. I haven't changed my scraper and the data format seems to be the same as my previously working JSON file. I'm not sure what could be causing the error. I would appreciate any advice and but I'd like to avoid using json.loads() and going through the file line by line. 

[
{"cards": ["Den Protector", "Duskwatch Recruiter", "Hidden Dragonslayer", "Jace, Vryn's Prodigy", "Sylvan Advocate", "Bounding Krasis", "Nissa, Vastwood Seer", "Reflector Mage", "Tireless Tracker", "Archangel Avacyn", "Dromoka's Command", "Collected Company", "Ojutai's Command", "Canopy Vista", "Evolving Wilds", "Forest", "Fortified Village", "Island", "Lumbering Falls", "Plains", "Port Town", "Prairie Stream", "Clip Wings", "Declaration in Stone", "Den Protector", "Dromoka's Command", "Hidden Dragonslayer", "Negate", "Lantern Scout", "Ojutai's Command", "Sigarda, Heron's Grace", "Tragic Arrogance"], "quantities": ["\n1\n", "\n4\n", "\n1\n", "\n3\n", "\n4\n", "\n4\n", "\n1\n", "\n4\n", "\n3\n", "\n2\n", "\n3\n", "\n4\n", "\n1\n", "\n3\n", "\n4\n", "\n3\n", "\n1\n", "\n2\n", "\n4\n", "\n3\n", "\n1\n", "\n4\n", "\n2\n", "\n2\n", "\n1\n", "\n1\n", "\n1\n", "\n3\n", "\n1\n", "\n1\n", "\n1\n", "\n2\n"], "name": ["Bant Company  by Kevin Sauls Deck"]},
{"cards": ["Deathcap Cultivator", "Duskwatch Recruiter", "Jace, Vryn's Prodigy", "Rattlechains", "Sylvan Advocate", "Bounding Krasis", "Eldrazi Skyspawner", "Nissa, Vastwood Seer", "Reflector Mage", "Tireless Tracker", "Sigarda, Heron's Grace", "Declaration in Stone", "Dromoka's Command", "Nissa, Voice of Zendikar", "Collected Company", "Canopy Vista", "Evolving Wilds", "Forest", "Island", "Lumbering Falls", "Plains", "Prairie Stream", "Yavimaya Coast", "Dispel", "Clip Wings", "Declaration in Stone", "Den Protector", "Felidar Cub", "Negate", "Stratus Dancer", "Lantern Scout", "Ojutai's Command", "Archangel Avacyn"], "quantities": ["\n1\n", "\n4\n", "\n1\n", "\n2\n", "\n4\n", "\n4\n", "\n2\n", "\n1\n", "\n4\n", "\n2\n", "\n1\n", "\n1\n", "\n3\n", "\n1\n", "\n4\n", "\n4\n", "\n4\n", "\n3\n", "\n2\n", "\n4\n", "\n3\n", "\n4\n", "\n1\n", "\n2\n", "\n1\n", "\n1\n", "\n2\n", "\n1\n", "\n2\n", "\n1\n", "\n1\n", "\n2\n", "\n2\n"], "name": ["Bant Company  by Jessy Hefner Deck"]}
...
]
Tony Situ
  • 13
  • 4
  • Even though you might have valid json python's `json.load` doesn't support decoding multiple json objects at once. http://stackoverflow.com/a/21058946/322909 – John Apr 27 '16 at 04:59
  • @John I was under the impression that json.load() can handle decoding multiple json dictionaries as long as they are enclosed within a list (I have enclosed a snippet above as an example). This is the same format of my previous JSON file which worked fine with earlier calls to python's json.load(), is there any other reason why it could error? – Tony Situ Apr 27 '16 at 05:16
  • if it's enclosed by a list you should be fine. You probably have a typo somewhere, something like `json.loads('{"test":1}}')`. Is there anyway you could post the contents of your file or is it too big? – John Apr 27 '16 at 05:22
  • Here are links to the old and new json files. I appreciate the help John! old: http://pastebin.com/einQLnGB new : http://pastebin.com/eH2J9k1R – Tony Situ Apr 27 '16 at 05:33
  • I copy and pasted both pastebins into files on my desktop and was able to load them just fine ... – John Apr 27 '16 at 05:40
  • I found the error, I was handling namespaces incorrectly in my framework and I was trying to parse a JSON file that I was unknowingly appending to. Thanks for being my rubber duck John! – Tony Situ Apr 27 '16 at 06:26

1 Answers1

0

Solved, the error was due to a bug in my framework's namespace not the actual JSON file. Refer to comments above.

Tony Situ
  • 13
  • 4