1

I have a code, in which I want let the user to pass through stdin three values, each of type float, Fraction or int, so I can't apply ast.literal_eval on input to get result I want.

I read Eval really is dangerous, showing

  • how eval's weaknesses can be used by person who gives input, and
  • some ways how to make eval invokes more safe.

I applyed author's advices and wrote this code, replacing default eval:

import builtins

class DoubleUnderscoreInEval(ValueError):
    pass

def eval(expression, globals={}, locals=None):
    if '__' in expression:
        raise DoubleUnderscoreInEval('Using __ in eval is not allowed for safety reasons.')
    else:
        if '__builtins__' not in globals:
            globals['__builtins__']={}
        return builtins.eval(expression, globals, locals)

Is using builtins.eval through my code completely safe?
If no, can I make eval invokes completely safe? (string circumventing my restrictions is encouraged)
If yes, how? If no, what can I use instead?

I'm beginner in Python, so any comments telling how can I improve my code are encouraged.

My whole code in online compiler

GingerPlusPlus
  • 5,336
  • 1
  • 29
  • 52
  • 3
    [Betteridge's law of headlines](http://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines) springs to mind ;-) – Zero Piraeus Sep 20 '14 at 18:53

1 Answers1

5

No, your eval() implementation is not safe. Other objects can give access to the __builtins__ mapping too.

See eval is dangerous by Ned Batchelder for some examples.

For example, the following string can cause a segfault:

s = """
(lambda fc=(
    lambda n: [
        c for c in 
            ().__class__.__bases__[0].__subclasses__() 
            if c.__name__ == n
        ][0]
    ):
    fc("function")(
        fc("code")(
            0,0,0,0,"KABOOM",(),(),(),"","",0,""
        ),{}
    )()
)()
"""

Merely disallowing double underscores is not going to be enough, I fear. Some enterprising soul will find a method of re-combining double underscores in a creative way you didn't anticipate and you'll find yourself hacked anyway.

The ast.literal_eval() method uses AST parsing and custom handling of the parsetree to support built-in types. You could do the same to by adding support for 'safe' callables:

import ast

def literal_eval_with_callables(node_or_string, safe_callables=None):
    if safe_callables is None:
        safe_callables = {}
    if isinstance(node_or_string, str):
        node_or_string = ast.parse(node_or_string, mode='eval')
    if isinstance(node_or_string, ast.Expression):
        node_or_string = node_or_string.body
    try:
        # Python 3.4 and up
        ast.NameConstant
        const_test = lambda n: isinstance(n, ast.NameConstant)
        const_extract = lambda n: n.value
    except AttributeError:
        # Everything before
        _const_names = {'None': None, 'True': True, 'False': False}
        const_test = lambda n: isinstance(n, ast.Name) and n.id in _const_names
        const_extract = lambda n: _const_names[n.id]
    def _convert(node):
        if isinstance(node, (ast.Str, ast.Bytes)):
            return node.s
        elif isinstance(node, ast.Num):
            return node.n
        elif isinstance(node, ast.Tuple):
            return tuple(map(_convert, node.elts))
        elif isinstance(node, ast.List):
            return list(map(_convert, node.elts))
        elif isinstance(node, ast.Dict):
            return dict((_convert(k), _convert(v)) for k, v
                        in zip(node.keys, node.values))
        elif const_test(node):
            return const_extract(node)
        elif isinstance(node, ast.UnaryOp) and \
             isinstance(node.op, (ast.UAdd, ast.USub)) and \
             isinstance(node.operand, (ast.Num, ast.UnaryOp, ast.BinOp)):
            operand = _convert(node.operand)
            if isinstance(node.op, ast.UAdd):
                return + operand
            else:
                return - operand
        elif isinstance(node, ast.BinOp) and \
             isinstance(node.op, (ast.Add, ast.Sub)) and \
             isinstance(node.right, (ast.Num, ast.UnaryOp, ast.BinOp)) and \
             isinstance(node.right.n, complex) and \
             isinstance(node.left, (ast.Num, ast.UnaryOp, astBinOp)):
            left = _convert(node.left)
            right = _convert(node.right)
            if isinstance(node.op, ast.Add):
                return left + right
            else:
                return left - right
        elif isinstance(node, ast.Call) and \
             isinstance(node.func, ast.Name) and \
             node.func.id in safe_callables:
            return safe_callables[node.func.id](
                *[_convert(n) for n in node.args],
                **{kw.arg: _convert(kw.value) for kw in node.keywords})
        raise ValueError('malformed string')
    return _convert(node_or_string)

The above function adapts the implementation of ast.literal_eval() to add support for specific registered callables. Pass in a dictionary naming Fraction:

>>> import fractions
>>> safe_callables = {'Fraction': fractions.Fraction}
>>> literal_eval_with_callables('Fraction(1, denominator=2)', safe_callables)
Fraction(1, 2)

The above method whitelists rather than blacklists what will be handled. Calling Fraction is allowed, and directly controlled exactly what object will be called, for example.

Demo:

>>> samples = '''\
... 2, -9, -35
... 4, -13, 3
... -6, 13, 5
... 5, -6, 6
... -2, 5, -3
... 4, 12, 9
... 0.5, 1, 1
... 1, -0.5, -0.5
... 0.25, Fraction(-1, 3), Fraction(1, 9)'''.splitlines()
>>> safe_callables = {'Fraction': fractions.Fraction}
>>> for line in samples:
...     print(literal_eval_with_callables(line, safe_callables))
... 
(2, -9, -35)
(4, -13, 3)
(-6, 13, 5)
(5, -6, 6)
(-2, 5, -3)
(4, 12, 9)
(0.5, 1, 1)
(1, -0.5, -0.5)
(0.25, Fraction(-1, 3), Fraction(1, 9))

The above should work on at least Python 3.3 and up, possibly earlier. Python 2 would require some more work to support unicode strings and not break on ast.Bytes.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • The following string contains double underscore, so it's going to be rejected by my implementation. – GingerPlusPlus Sep 20 '14 at 18:43
  • 3
    @GingerPlusPlus: There are ways around that too. Bottom line: don't use `eval()`. – Martijn Pieters Sep 20 '14 at 18:45
  • Solution you recommended seems nice, but unfortunately [it doesn't work](http://rextester.com/QFPHKR28878) with [my code](http://rextester.com/VJBIP29523) – GingerPlusPlus Sep 20 '14 at 20:19
  • @GingerPlusPlus: I assumed Python 2, I switched over to an adapter version from Python 3 instead. – Martijn Pieters Sep 20 '14 at 21:37
  • @GingerPlusPlus: your simplification pushes a mutable into the defaults. Sort-of okay in this case, a bad idea in most cases. I made that choice explicitly. – Martijn Pieters Sep 21 '14 at 10:49
  • @Martijin: why it's bad idea in most cases? – GingerPlusPlus Sep 21 '14 at 10:53
  • 1
    @GingerPlusPlus: see ["Least Astonishment" in Python: The Mutable Default Argument](http://stackoverflow.com/q/1132941) – Martijn Pieters Sep 21 '14 at 10:53
  • Replaced `ast.NameConstant` using `ast.Name` and [finally it works](http://rextester.com/MZDMJ30071) – GingerPlusPlus Sep 21 '14 at 12:01
  • @GingerPlusPlus: what Python 3 version are you running at all? This was tested with Python 3.4. – Martijn Pieters Sep 21 '14 at 12:30
  • @GingerPlusPlus: the `ast.NameConstant` represents `None`, `True`, `False`, `NotImplemented`, or `Ellipsis` (e.g. singleton named constants in the Python language); you cannot just replace that with `ast.Name`, which doesn't even have a `node.value` attribute. `ast.Name` represents an identifier, like `Float`, which is what he handle with the `ast.Call` node further on in the code. – Martijn Pieters Sep 21 '14 at 12:54
  • @GingerPlusPlus: it appears the Python version running on `http://rextester.com/` is *too old* to support named constants. You'd remove it altogether or replace it with how the [Python 2 version of `ast.literal_eval()`](https://bitbucket.org/mirror/cpython/src/af1351800c7a342d752546b63195b62efe9e8347/Lib/ast.py?at=2.7#cl-40) handles `ast.Name`; by looking up the name in a `_safe_names` mapping. – Martijn Pieters Sep 21 '14 at 12:57
  • @GingerPlusPlus: I've updated the method to work on Python 3.3 and 3.4. – Martijn Pieters Sep 21 '14 at 14:54
  • I have no idea why, but even that [doesn't work when passed something different than positive `float`/`int`](http://rextester.com/IUWGU37510)! `AttributeError` error is expected, so it occurs in `try`, and below is fitting `except`, but it becomes uncaught, I have no idea why. Link leads to modyfied version of my program: I commented out `except Exception` in main part of the program, to allow seeing traceback, and putted in _stdin_ 2 examples of input, which causes this unexpected behavior. Anyway, at the moment I'm working under Python3.3, but in near future possibly I'll change it to 3.4. – GingerPlusPlus Sep 21 '14 at 18:37
  • @GingerPlusPlus: no, the error is entirely mine. One more line added, and properly tested this time. See http://rextester.com/IUWGU37510 – Martijn Pieters Sep 21 '14 at 20:10
  • Finally it works woth no problems, thank you very much. – GingerPlusPlus Sep 22 '14 at 09:19