Not able to parse JSON (which is not nested JSON) object coming from socket in python

Question

New in python, I am reading JSON object from the server, size of JSON object is not fix. I am getting data from the server as per the buffer size given in socket.recv(1024). How to check that JSON object from server-socket received is full/complete, Because while parsing that JSON I am getting an error. Please note that my JSON object is not nested.

**** Sample code ****

def get_data():
    s = socket.socket()
    host = 'IP_Address'
    port = 'Port_Number'

    # connection to hostname on the port.
    s.connect((host, port))
    msg=''
    while(True):        
        msg = s.recv(1024)
        print(msg.decode('ascii'))   
        jsonObject=json.loads(msg.decode('ascii'))     

    s.close()

Below is the error

Traceback (most recent call last):

  File "d:/xxxxxxxxxxxxx/Python_Test.py", line 26, in <module>
    get_data()

  File "d:/xxxxxxxxxxxxx/Python_Test.py", line 20, in get_data
    temp=json.loads(msg.decode('ascii'))

  File "xxxxxxxxxxxxx\Python\Python37\lib\json\__init__.py", line 348, in loads
    return _default_decoder.decode(s)

  File "xxxxxxxxxxxxx\Python\Python37\lib\json\decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)

json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 777)

Please show us what is `msg` in your code (print it before the line that has error) — Mojtaba Kamyabi, Mar 13 '19 at 07:10
You print what you have received. Could you share it here, so that we can better understand what happens? — Serge Ballesta, Mar 13 '19 at 07:14
@SergeBallesta you are right. I delete that comment :) May be the problem is because of recieving only 1024 bytes in every loop and the size of the json is bigger than that — Mojtaba Kamyabi, Mar 13 '19 at 07:16
@shotgunner below is the message.{"divice_id":"37843","slot":"2019-03-13 11:49:58.255","1":"237.51","2":"240.79","3":"241.48","4":"10.28","62":"383.0"} {"divice_id":"33295","slot":"2019-03-13 11:49:58.33","1":"238.88","2":"240.71","3":"241.05","4":"165.9","5":"187.63","6":"162.47","7":"3630 — Amit, Mar 13 '19 at 07:17
Yes size of my Json object is bigger and also getting more than one json object in one msg. client is getting msg as per the buffer size. — Amit, Mar 13 '19 at 07:21
@shotgunner so can you tell me please how can I loop the `recv` to recive complete JsonObject and one message at a time — Amit, Mar 13 '19 at 07:24
@Amit is your json not nested ? like this : `{"device_id": {"another_item": 2}}` — Mojtaba Kamyabi, Mar 13 '19 at 07:36

Mojtaba Kamyabi · Accepted Answer · 2019-03-13T10:27:08.797

1

You recieve 1024 bytes in every loop and if your json object is bigger than that you have to handle uncompleted json string.
Also you may have two json object in 1024 bytes or even more. you can change your code to below code

def get_data():
    s = socket.socket()
    host = 'IP_Address'
    port = 'Port_Number'
    s.connect((host, port))
    msg=''
    while True:        
        r = s.recv(1024)
        msg += r.decode('ascii')
        while True:
            start = msg.find("{")
            end = msg.find("}") 
            if start==-1 or end==-1:   # if can not find both { and } in string  
                break
            jsonObject=json.loads(msg[start:end+1])  # only read { ... } and not another uncompleted data
            #  do whatever you want with jsonObject here
            msg = msg[end+1:]
    s.close()

NOTE : this code work correct only if you have not any nested json in your data (like this: {"device_id": {"another_json": "something"}})

edited Mar 13 '19 at 10:27

answered Mar 13 '19 at 07:39

Mojtaba Kamyabi

3,440
3
29
50

Thank you for your reply, when I tried to execute your code, I got below error UnboundLocalError: local variable 'r' referenced before assignment. So where should I declare that r inside while loop or outside while loop – Amit Mar 13 '19 at 09:36
And when I declare that r='' below msg='' and above while loop I got below error r += s.recv(1024) TypeError: can only concatenate str (not "bytes") to str – Amit Mar 13 '19 at 09:39
@Amit this was a type in `r += ` I fix the code check it now – Mojtaba Kamyabi Mar 13 '19 at 10:22
Thank you so much. I got the required output. – Amit Mar 13 '19 at 10:42

score 0 · Answer 2 · answered Mar 13 '19 at 07:52

0

You need to wait till the complete json is received and then parse it, something like this should work:

msg = ''
while(True):        
    response = s.recv(1024)
    if not response:
        break
    msg += response
jsonObject=json.loads(msg.decode('ascii'))

answered Mar 13 '19 at 07:52

Ankit Jaiswal

22,859
5
41
64

Hi Ankit, what if in the case where I received two JSON obj in the same msg. – Amit Mar 13 '19 at 10:01
@Amit in that case `json.JSONDecoder.raw_decode` is the right way. See this https://stackoverflow.com/a/50384432/380829 – Ankit Jaiswal Mar 18 '19 at 07:01

Not able to parse JSON (which is not nested JSON) object coming from socket in python

2 Answers2