Using eval() with Undefined Variables

Question

I want to create a Python script that takes in a string representation of a dictionary and outputs a list of tuples representing the items of the dictionary. The rub is, I want it to take in variables that have not been defined. An example illustrates this best:

Input: {'test': test}

Output: [('test': test)]

I've created some code to read in the dictionary and define the variables that haven't yet been defined, but when I eval() the result, it substitutes in the actual value of the variable rather than the variable name.

Here is the code:

import sys
import re

if __name__ == "__main__":
    instr = sys.argv[1]
    success = False
    while not success:
        try:
            indict = eval(instr)
            success = True
        except NameError, e:
            defname = re.search("name '([^\']*)' is not defined", str(e))
            locals()[defname.group(1)] = 0
    print indict

I was hoping, for the above defined input value, that the dict it printed out would match the input string perfectly, but instead it has substituted in the value of 0 for test. Thus the script prints out:

{'test': 0}

I have tried ast.literal_eval, but it throws a ValueError: malformed string for any variable name in the literal, even if it's been defined.

Thus, my question is: Is there a way to convert the input dictionary without losing the variable names?

score 2 · Answer 1 · answered Aug 11 '11 at 14:10

2

I would use the ast module, but not literal_eval. Walk the tree, when you find a variable reference, just insert the name of the variable into the output.

answered Aug 11 '11 at 14:10

Ned Batchelder

364,293
75
561
662

According to the definition of `ast.walk` (and my own experimentation), the nodes are given back in no particular order, so I cannot output a reliable representation of the input string with it. – Brent Newey Aug 11 '11 at 14:49
`ast.walk` isn't what you want. You need to visit each node in the tree. See http://stackoverflow.com/questions/1515357/simple-example-of-how-to-use-ast-nodevisitor for an example. – Ned Batchelder Aug 11 '11 at 14:53

score 1 · Answer 2 · answered Aug 11 '11 at 14:55

1

You can trick eval into giving you what you want:

class Reflector(object):
    def __getitem__(self, name):
        return name

s = "{'test':test}"

print eval(s, globals(), Reflector())

produces:

{'test': 'test'}

The usual caveats about the dangers of eval hold though: if you have any doubts about the source of these strings, you are opening yourself up for hacking.

answered Aug 11 '11 at 14:55

Ned Batchelder

364,293
75
561
662

I have no doubts about the source. – Brent Newey Aug 11 '11 at 15:03
This is interesting, but only gets me to the point of having a string with the name of the variable instead of the variable. – Brent Newey Aug 11 '11 at 15:49

score 0 · Answer 3 · answered Aug 11 '11 at 14:17

I think the problem you are having is that you want to evaluate the name of a variable instead of the value of that variable. In particular, {'test':test} would never be the output of printing a dictionary because test is not a value in python, it is possibly the name of a variable. The closest you can get is assigning

locals()[defname.group(1)] = defname.group(1)

to get

{'test':'test'}

score 0 · Answer 4 · answered Aug 11 '11 at 15:03

I think a reasonable workaround is to pass eval something that preserves undefined variables in a useful way. You can pass any dict subclass to eval, and you can override the __missing__ method of dict to return an 'undefined variable' object:

>>> class Var(object):
...     def __init__(self, name):
...         self.name = name
...     def __repr__(self):
...         return "Var(%r)" % (self.name,)
... 
>>> class UndefSubst(dict):
...     def __missing__(self, key):
...         return Var(key)
... 
>>> foo = 'bar'
>>> eval("{'baz': quux, 1: foo}", UndefSubst(locals()))
{1: 'bar', 'baz': Var('quux')}

score 0 · Accepted Answer · answered Aug 11 '11 at 15:54

I solved this problem by substituting in unique strings for the names of the variables. This produces the desired result:

import sys
import re

if __name__ == "__main__":
    instr = sys.argv[1]
    subs = []
    success = False
    while not success:
        try:
            indict = eval(instr)
            success = True
        except NameError, e:
            defname = re.search("name '([^\']*)' is not defined", str(e)).group(1)
            subs.append(defname)
            locals()[defname] = '//substitute{%d}' % (len(subs) - 1)
    outlist = indict.items()
    outstr = str(outlist)
    for i, sub in enumerate(subs):
        outstr = outstr.replace("'//substitute{%d}'" % i, sub)
    print outstr

Input: {"test": test}

Output: [('test', test)]

Input: {"test": [{"test": test}, {word: "word"}]}

Output: [('test', [{'test': test}, {word: 'word'}])] (note that this is desirable, I don't want the inner dictionary zipped).

The minor downside is that I can never have whatever substitution string I elect to use anywhere in the input string, but I am reasonably certain this will not be an issue for what I wish to use this script for.

Using eval() with Undefined Variables

5 Answers5