How to include \n with python -m json.tool

Question

I'm trying to create a json file via shell, however, new lines are now allowed and through an error.

Invalid control character at: line 5 column 26 (char 87) which points to \n

echo '{
    "param1": "asdfasf",
    "param2": "asffad",
    "param3": "asdfsaf",
    "param4": "asdfasf\nasfasfas"
}' | python -m json.tool > test.json

Assuming I'd like to preserve new lines, how can I get this to put a json file?

UPDATE:

I'm thinking it has something to do with strict mode for python's json encoder/decoder.

If strict is False (True is the default), then control characters will be allowed inside strings. Control characters in this context are those with character codes in the 0-31 range, including '\t' (tab), '\n', '\r' and '\0'.

https://docs.python.org/2/library/json.html

How can strict mode be set to False from within python -m json.tool?

`asdfasf` has no quotes around it, otherwise it works for me — vaultah, Aug 11 '15 at 14:42
Sorry, I fixed the quotes, but it still does not work for me and outputs the same error message. — Raphael Rafatpanah, Aug 11 '15 at 14:43

Padraic Cunningham · Accepted Answer · 2015-08-11T15:15:35.150

Escaping the \ seems to do the trick:

echo  '{
    "param1": "asdfasf",
    "param2": "asffad",
    "param3": "asdfsaf",
    "param4": "asdfasf\\nasfasfas"
}' | python -m json.tool > test.json

It creates valid json:

with open('/home/test.json', 'rU') as f:
    js = json.load(f)
    print(js)
    print(js["param4"])

Output:

{'param1': 'asdfasf', 'param3': 'asdfsaf', 'param2': 'asffad', 'param4': 'asdfasf\nasfasfas'}
asdfasf
asfasfas

score 1 · Answer 2 · answered Aug 11 '15 at 15:05

1

zsh is replacing \n with a proper carriage return. You could escape it or use heredoc style instead:

python -m json.tool > test.json << EOF
{
"param1": "asdfasf",
"param2": "asffad",
"param3": "asdfsaf",
"param4": "asdfasf\nasfasfas"
}
EOF

answered Aug 11 '15 at 15:05

Alastair McCormack

26,573
8
77
100

Again, this results in a json file with the string `\n` in the json itself. – Raphael Rafatpanah Aug 11 '15 at 15:08
But AFAIK and according to http://stackoverflow.com/questions/2392766/multiline-strings-in-json, actual newlines are illegal in JSON. – Alastair McCormack Aug 11 '15 at 15:10
For example, if you create the input JSON in a file with a proper line feed and cat that to your Python command, you'll get the same error :) – Alastair McCormack Aug 11 '15 at 15:15
Thank you I did not realize this earlier. – Raphael Rafatpanah Aug 11 '15 at 15:19
Also, strict mode relates to serialising but your error will be occurring deserialisation. Using `json.load(input)` throws a similar error when it finds an actual carriage return :) – Alastair McCormack Aug 11 '15 at 15:22

How to include \n with python -m json.tool

2 Answers2