Python: eval string with \n at the end

Question

How can I perform a eval on a string with \n ?

Why does this not work?

a = eval('"hello \n"')
In [70]: eval("\"hello \n\"")
  File "<string>", line 1
    "hello
          ^
SyntaxError: EOL while scanning string literal

Whereas this does

a = "hello \n"

My use case is that a script executed through subprocess is outputing a dictionary as a string of which I am capturing the stdout of it and I would like to perform an eval on it.

'''[
     { "hello": "the description of this is\' \n"}
]'''

Do you know what `eval` does? What exactly would you even expect `eval("hello")` to do, even without the newline character? Your second example is just assigning a string to a variable. — Cory Kramer, Jan 08 '16 at 14:43
I would expect it to return a string p.s. its not eval("hello") its eval('"hello"') — Har, Jan 08 '16 at 14:44
What's the use case? Why do you need to call eval on the string? Why does it need to have '\n'? If you share more about the problem we can help you more — Amaury Medeiros, Jan 08 '16 at 14:47
And how does your subprocess produce that output? Also, have you considered using JSON instead? — Martijn Pieters, Jan 08 '16 at 15:00
It simply does a pprint.pprint(my_dictionary) a dictionary to stdout. Good point, JSON would have been a better option :( — Har, Jan 08 '16 at 15:04
@Har: `pprint()` output should just work; something else is turning those literal `\` and `n` characters into `\n` before you pass them to `eval()`. — Martijn Pieters, Jan 08 '16 at 15:17
@Martijn Pieters are you sure? pprint.pprint("Hello world \n") returns 'Hello world \n' which wouldnt be able to be evaled. From TigerhawkT3 post I think I should be performing an encode before output on the string so "Hello world\n".encode("string_escape") and then in the capturing program perform a decode after the capture. — Har, Jan 08 '16 at 15:22
I see your point, pprint.pprint(r"Hello world\n") would work but not when the string is in a variable, only when it is in a literal. — Har, Jan 08 '16 at 15:24
@Har: it doesn't return anything; it writes to `stdout`. But that string written to stdout is valid Python code. *Just don't type it into Python again*, because *then* you ask Python to interpret the `\n` character sequence. — Martijn Pieters, Jan 08 '16 at 15:24
@Har: `eval(pprint.pformat('hello\n'))` works just fine, because `pprint.pformat()` returns the resulting pretty-printed string rather than write it to stdout. — Martijn Pieters, Jan 08 '16 at 15:25
@Har: If you use `pprint.pprint()` and read from stdout in a parent process, the `\` and `n` remain separate characters too, and can be passed to `eval()`. Still a bad idea, because it is slow, and there are better alternatives. `ast.literal_eval()` limits the code loading only Python literals (safe), but JSON is just much faster for this kind of exchange. — Martijn Pieters, Jan 08 '16 at 15:31
@MartijnPieters thank you very much, worked like magic :) that solved my problem. — Har, Jan 08 '16 at 15:32
@MartijnPieters what worked for me is pprint.pprint(pprint.pformat(mystring)) I cant seem to get a reliable result out of simply pprint.pprint-ing it , Ill keep JSON in mind for future projects :o) seems to be the right answer here. — Har, Jan 08 '16 at 15:34

score 6 · Answer 1 · answered Jan 08 '16 at 14:49

6

You need to escape the backslash.

>>> eval('"hello \n"')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    "hello
          ^
SyntaxError: EOL while scanning string literal
>>> eval('"hello \\n"')
'hello \n'
>>> print(eval('"hello \\n"'))
hello

>>>

Without that escape, Python will see this code (which is an obvious error):

"hello 
"

Rather than the desired code:

"hello \n"

answered Jan 08 '16 at 14:49

TigerhawkT3

48,464
6
60
97

I am confused as to what needs to be escaped and what doesn't, could you please clarify that? If for instance I have eval(' "hello's\n" ') it needs to be changed to eval(' "hello\'\\n" '), which characters need to be escaped? – Har Jan 08 '16 at 15:01
2

If the strings aren't coming from literals, as you mention in an edit, you might take a look at [this question](http://stackoverflow.com/questions/4020539/process-escape-sequences-in-a-string-in-python). – TigerhawkT3 Jan 08 '16 at 15:04

Martijn Pieters · Accepted Answer · 2016-01-08T15:44:27.587

If you want to specify a string that has a literal \ and n in it you either need to double the backslash, or use a raw string literal:

>>> '"hello\\n"'
'"hello\\n"'
>>> r'"hello\n"'
'"hello\\n"'

Such a string can then be evaluated as a Python expression containing a string literal:

>>> eval(r'"hello\n"')
'hello\n'

If your output is produced by a child process outputting the value with pprint.pprint(), you are doing more than just read that stream, as that produces perfectly valid Python syntax. Don't copy and paste that output into a Python interpreter, for example, because that'll just interpret the escape sequences directly (so before you pass it to eval()). If you are developing in an interpreter, you could use pprint.pformat() to produce a variable with the output, rather than write to stdout.

If you are trying to use Python repr() or pprint.pprint() output to pass data between systems, however, stop right there. Use a proper serialisation format instead, such as JSON. If that's not an option, at the very least use ast.literal_eval() to limit what your code accepts to only Python literals, and not arbitrary code (such as '__import__("os").system("rm -rf /")).

Could you also mention advice on pprint.pformat and also about encoding/decoding the string? '__import__("os").system("rm -rf /") is really a great example of why not to use external strings as if they are python code :) — Har, Jan 08 '16 at 15:41

Python: eval string with \n at the end

2 Answers2