-2

I'm parsing log files where each line is a json string. Here's two example, one works with json.loads (python 2.7 or python 3.7) and the other failed with a delimiter error. The only difference is the values of keys: "cache_response_code" and "server_response"

If the value is 000, it got the error, if it's a normal HTTP response, like "404" it works no problem. If the value is set to 0, it also works. It only fail if the value is 00 or 000 or any multiple 0 value.

Why ?

Example:

import json

line1 = '{"localtimestamp": "18/Mar/2020:13:39:04 -0400", "client_ip":"10.247.69.247", "cache_response":"ERR_INVALID_URL", "cache_response_code":404, "request_page":"http://10.247.135.122/", "server_response":404, "bytes":297, "user_agent":"-", "sessionId": "-", "origin_response_time": 0, "client_response_time": 0, "ssl_connection": 0, "referrer": "-"}'
line_json1 = json.loads(line1)

line2 = '{"localtimestamp": "18/Mar/2020:13:39:09 -0400", "client_ip":"10.247.69.247", "cache_response":"ERR_UNKNOWN", "cache_response_code":000, "request_page":"/", "server_response":000, "bytes":0, "user_agent":"-", "sessionId": "-", "origin_response_time": 0, "client_response_time": 5001, "ssl_connection": 0, "referrer": "-"}'
line_json2 = json.loads(line2)

Output in interpreter:

>>> import json
>>>
>>> line1 = '{"localtimestamp": "18/Mar/2020:13:39:04 -0400", "client_ip":"10.247.69.247", "cache_response":"ERR_INVALID_URL", "cache_response_code":404, "request_page":"http://10.247.135.122/", "server_response":404, "bytes":297, "user_agent":"-", "sessionId": "-", "origin_response_time": 0, "client_response_time": 0, "ssl_connection": 0, "referrer": "-"}'
>>> line_json1 = json.loads(line1)
>>>
>>> line2 = '{"localtimestamp": "18/Mar/2020:13:39:09 -0400", "client_ip":"10.247.69.247", "cache_response":"ERR_UNKNOWN", "cache_response_code":000, "request_page":"/", "server_response":000, "bytes":0, "user_agent":"-", "sessionId": "-", "origin_response_time": 0, "client_response_time": 5001, "ssl_connection": 0, "referrer": "-"}'
>>> line_json2 = json.loads(line2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting , delimiter: line 1 column 134 (char 133)
Gilbert
  • 13
  • 3
  • Does this answer your question? [Python json.loads ValueError, expecting delimiter](https://stackoverflow.com/questions/28223242/python-json-loads-valueerror-expecting-delimiter) – krmogi Oct 11 '21 at 02:11

3 Answers3

0

This seems to be working fine. I have just put the responses codes in double-quotes,

import json

line1 = '{"localtimestamp": "18/Mar/2020:13:39:04 -0400", "client_ip":"10.247.69.247", "cache_response":"ERR_INVALID_URL", "cache_response_code":"404", "request_page":"http://10.247.135.122/", "server_response":"404", "bytes":297, "user_agent":"-", "sessionId": "-", "origin_response_time": 0, "client_response_time": 0, "ssl_connection":0, "referrer": "-"}'
line_json1 = json.loads(line1)
line2 = '{"localtimestamp": "18/Mar/2020:13:39:09 -0400", "client_ip":"10.247.69.247", "cache_response":"ERR_UNKNOWN", "cache_response_code":"000", "request_page":"/", "server_response":"000", "bytes":0, "user_agent":"-", "sessionId": "-", "origin_response_time": 0, "client_response_time": 5001, "ssl_connection": 0, "referrer": "-"}' 
line_json2 = json.loads(line2)
KUNAL VERMA
  • 17
  • 4
0

Your str is not in good format:


# str like this will work fine
s1 = '"{\\"code\\":000}"'
json.loads(s1)

# works good if do a dumps() before loads()
json.loads(json.dumps(line2))
Andy Su
  • 131
  • 7
0

I've found an elegant solution to my problem. instead of json, I use ast

import ast

line1 = '{"cache_response_code":404, "referrer": "-"}'
line1_dict = ast.literal_eval(line1)

line2 = '{"cache_response_code":000, "referrer": "-"}'
line2_dict = ast.literal_eval(line2)

my real code is:

if files_dict:
    for file in files_dict.values():
        with open(file) as f:
            lines = f.readlines()
        for line in lines:
            line = ast.literal_eval(line)
Gilbert
  • 13
  • 3