open invalid json file in python

Question

I have a file with such non-valid json data (it's cut for clarity):

[
{
  "orderID": 90,
  "orderDate": '2017-05-10',  #issue №1
  "clientName": "Mr. Bean",
  "clientPhoneN": "123-4567",
  "orderContents": [
    {
      "productID": 05,        #issue №2
      "productName": "Bicycle",
      "quantity": 1,
      "price": 8000
    },
    {
      "productID": 23,
      "productName": "helmet",
      "quantity": 2,
      "price": 1000
    }
  ],
  "orderCompleted": true
}
]

I tried to open it in python and transform it to list of dictionaries, but with no success. Depending on the case I get different errors. It will take too much space to outline all my attempts and their ending errors.

I have two issues here with the file:

issue №1 - single quotes in orderDate value. it results with :

JSONDecodeError: Expecting value

issue №2 - zero leading productID. It results with:

JSONDecodeError: Expecting ',' delimiter

I can hardcode these issues, but I feel that it's not true pythonic way.

Is there an option of "pretty" opening and converting this data file to list of dictionaries?

Most probably I want to keep productID data typa as integer, but if it's impossible, str is ok too.

it's not valid json. have a look at [jsonlint](https://jsonlint.com/). — Lex Scarisbrick, Aug 10 '17 at 19:06
I know that it's invalid, I wrote it in the heading of my question. Anybody can open valid `json`. The very question is how to process _such_ `json` which is broken. — Dmitriy Fialkovskiy, Aug 10 '17 at 19:07
you are contradicting this as `json`, it is *close* to being JSON format, but needs to be modified before it ***is*** JSON, as it is currently simply "data". If you cannot modify the contents of the "data", you will need to use a custom parser. — chickity china chinese chicken, Aug 10 '17 at 19:15
If you can modify the contents of the data, you could create valid JSON using double quotes for strings fix issue #1 https://stackoverflow.com/a/4162651/1248974 — chickity china chinese chicken, Aug 10 '17 at 19:17
@downshift, if to talk very strictly, you're correct, of course, this isn't `json`, but it meant to be it=) — Dmitriy Fialkovskiy, Aug 10 '17 at 19:19
Ah, thankfully at least *someone* knew what you were really asking for! :-) — chickity china chinese chicken, Aug 10 '17 at 19:21
@AlexEshoo, I don't know _what_ precisely outputs such data, but I needed to open it=) — Dmitriy Fialkovskiy, Aug 10 '17 at 19:23
I digress Mr. @DmitriyFialkovskiy, your question was *very* understandable, I must have misread and only ***I*** am the one who misinterpreted *your* very clear and understandable question, Sir. My sincere apologies. — chickity china chinese chicken, Aug 10 '17 at 19:29
@downshift, is it a jokish answer or you're taliking seriuos?) I really can't figure out — Dmitriy Fialkovskiy, Aug 10 '17 at 19:33
@DmitriyFialkovskiy, I am talking serious, I should not have made my incorrect comments, you are correct, I was incorrect and I genuinely and sincerely apologize. Your question has been answered. My apologies for confusing you and for me being confused and not understanding your question. — chickity china chinese chicken, Aug 10 '17 at 19:36
Can someone (who did it) tell me why he/she put downvote on my question? I want to take your feedback into account for the future. — Dmitriy Fialkovskiy, Aug 10 '17 at 19:40

score 2 · Accepted Answer · answered Aug 10 '17 at 19:15

Try demjson package:

from demjson import decode
decode("""[
{
  "orderID": 90,
  "orderDate": '2017-05-10',
  "clientName": "Mr. Bean",
  "clientPhoneN": "123-4567",
  "orderContents": [
    {
      "productID": 05,
      "productName": "Bicycle",
      "quantity": 1,
      "price": 8000
    },
    {
      "productID": 23,
      "productName": "helmet",
      "quantity": 2,
      "price": 1000
    }
  ],
  "orderCompleted": true
}
]""")

You'll get:

[{'clientName': 'Mr. Bean',
  'clientPhoneN': '123-4567',
  'orderCompleted': True,
  'orderContents': [{'price': 8000,
    'productID': 5,
    'productName': 'Bicycle',
    'quantity': 1},
   {'price': 1000, 'productID': 23, 'productName': 'helmet', 'quantity': 2}],
  'orderDate': '2017-05-10',
  'orderID': 90}]

I knew that there's something easy to apply=) thanks, Huang! — Dmitriy Fialkovskiy, Aug 10 '17 at 19:18
For those curious why this module works as opposed to the standard library `json`: "In non-strict mode it can also deal with slightly non-conforming input that is more JavaScript than JSON (such as allowing comments)." — Alex, Aug 10 '17 at 19:20

open invalid json file in python

1 Answers1