Python find python strings in string

Question

Ok,

If I have a string, say x = 'Hello World!', how can I parse python strings from that? I know that I can use regex like "[^"]*", but how I can parse all valid python strings? Solution doesn't have to be regex, but if that is possible, it's great.

For example:

x = 'Hello World!' => Hello World!
x = '\'Stack Overflow\'' => \'Stack Overflow\'
x = 'x=\"x=\'Python\n\'\"' => x=\"x=\'Python\n\'\"

Sorry, if I cannot explain it clearly, but this is not easy as non-native speaker.

Question unclear - your examples look like you want to strip the surrounding quotes out? Which aren't actually part of the string data itself, they define a string literal. — wkl, May 21 '14 at 13:52
presumably a "python string" is any character sequence matching the `stringliteral` rule described [here](https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals). — Kevin, May 21 '14 at 13:52
https://docs.python.org/2/reference/lexical_analysis.html#strings — Hannes Karppila, May 21 '14 at 13:53
Shouldn't the second one return `\'Stack Overflow\'`? You want to apply your regex to the literal text `x = 'Hello World!'`, right, not to `Hello World!`? — Robin, May 21 '14 at 14:06
Here's what you need: http://code.activestate.com/recipes/475109/ — John Smith Optional, May 21 '14 at 14:11

score 5 · Accepted Answer · edited May 23 '17 at 11:56

well, the simplest way would be to use ast.literal_eval():

>>> literal_eval(r"'Hello World!'")
'Hello World!'
>>> literal_eval(r"'\'Stack Overflow\''")
"'Stack Overflow'"
>>> literal_eval(r"""'x=\"x=\'Python\n\'\"'""")
'x="x=\'Python\n\'"'

but if you want to extract python strings from a string containing one ore several full python statements, you can do:

def get_string(s):
    for it in ast.walk(ast.parse(s)):
        if isinstance(it, ast.Str):
            yield it.s

here is the results:

>>> for i in get_string(r"'Hello World!'"): print i
... 
Hello World!

for the following match, to get the result you're expecting, you need to have your string setup as a raw string:

>>> for i in get_string("x = '\'Stack Overflow\''"): print i
... 
'Stack Overflow'
>>> for i in get_string(r"x = '\'Stack Overflow\''"): print i
... 
\'Stack Overflow\'

for the last match, to get the result you're expecting you need to have the inner string setup as a raw string:

>>> for i in get_string(r"""x = 'x=\"x=\'Python\n\'\"'"""): print i
... 
x="x='Python
'"
>>> for i in get_string(r"""x = r'x=\"x=\'Python\n\'\"'"""): print i
... 
x=\"x=\'Python\n\'\"

In the end, even though a non-regular regex can do the job, it'll always be a better option to actually use the parser that is used to parse python to parse python strings, because you'll be using the same tool being used to create and parse python strings!

It raises/throws an AttributeError: 'str' object has no attribute '_fields' (http://pastebin.com/4CKGE1g0) — Hannes Karppila, May 21 '14 at 14:52
sorry, I forgot the `ast.parse(s)` in the `get_string()` function. My bad. Corrected in the answer! — zmo, May 21 '14 at 14:53

Python find python strings in string

1 Answers1