Reading a file in python: json.decoder.JSONDecodeError

Question

I have a json file.

with open('list.json', "r") as f:
    r_list = json.load(f)

crashes with:

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0

I checked the schema online and the schema works.

The schema is very simple:

{"foo": [
{"name": "AAA\u2019s BBB CCC", "url": "/foome/foo"}
]}

Tried to play with:

file encoding
Try a dummy file

.. run out of ideas - is it something where ´json.load´ expects a binary?

Edit 1 Code works in a plain file, does not work in the scrapy class

import scrapy
from scrapy.selector import Selector
from scrapy.http import HtmlResponse
import json

class myScraper(scrapy.Spider):
    name="testScraper"

    def start_requests(self):        
        with open('test.json') as f:
            self.logger.info(f.read()) #shows the file content
            r_list = json.load(f) # breaks with the error msg
            yield "foo"

    def parse(self, response):
        self.logger.info("foo")

'test.json'

{
    "too": "foo"
}

Try `json.loads(f.read())`. Otherwise, if your file really says what you claim it says, and the path is the right file, then that program should work perfectly. — miike3459, Dec 28 '18 at 22:12
´json.loads(f.read())´ results in ´AttributeError: 'str' object has no attribute 'read'´ — NoIdeaHowToFixThis, Dec 28 '18 at 22:16
Putting that "schema" into a file and running your code on it works like a charm. Please come up with a [mcve] otherwise you'll just keep gathering answers that completely miss the problem using elaborate guesswork. — Andras Deak -- Слава Україні, Dec 28 '18 at 22:38
I created a new project. It works. The only difference is that I have the exact lines of code in a scrapy class. Will edit the question. — NoIdeaHowToFixThis, Dec 28 '18 at 22:58
Your `f.read()` in the logger call exhausts the iterator (reads the whole file). Subsequent reads from the same file handle won't yield you anything. Bind the file contents to a string and use that twice: `content = f.read(); self.logger.info(content); r_list = json.loads(content)`. The other option is to rewind your file handle to the beginning, but there's no need to read the same data twice. — Andras Deak -- Слава Україні, Dec 28 '18 at 23:44
One possible duplicate among many: https://stackoverflow.com/questions/3906137/why-cant-i-call-read-twice-on-an-open-file — Andras Deak -- Слава Україні, Dec 28 '18 at 23:49

corn3lius · Answer 1 · 2019-01-02T14:02:18.463

Most likely your file is empty.

Example:

https://repl.it/@mark_boyle_sp/SphericalImpressiveIrc

updated:

Your iterator is exhausted as also discussed in the comments. Since you log the files contents the iterator is at the end of the file. (looks like an empty file, hence the exception)

Reset the iterator or read the contents to a local value and operate on that.

json_str = f.read()
self.logger.info(json_str) #shows the file content
r_list = json.loads(json_str)

updated again

(I assume) The scrapy issue you are having is in the parse method? The response body is a bytes object you will need to decode it and use loads on the resulting string like so :

def parse(self, response):
    self.logger.info("foo")
    resp_str = response.body.decode('utf-8') 
    self.logger.info(resp_str) #shows the response
    r_list = json.loads(json_str)

Reading a file in python: json.decoder.JSONDecodeError

1 Answers1