Python: How to use ast.literal_eval() for sets in set?

Question

Sets are not hashable as they are mutable. But is there a way to still use literal_eval on {1, 2, {3, 4}}? I just want to know that the outer structure is a set, I don't care about the inner type, but sets within sets are possible inputs.

Update:

The input is read from a file as string.

Is `{1, 2, {3, 4}}` stored as a string? (If yes, then you can't) — Bhargav Rao, Dec 27 '16 at 16:12
Trying on the latest stable Python3.6: `{1, 2, {3, 4}}` is not a malformed string and is an acceptable input, but sets are still non-hashable, `frozensets` might be the way to approach the problem. — alecxe, Dec 27 '16 at 16:14

score 2 · Accepted Answer · answered Dec 28 '16 at 08:43

You can hack ast.literal_eval to make it return a frozenset when it sees a set. Here is how to do:

search where the library for your Python installation is
it contains the file ast.py that contains the function literal_eval
copy that function (using a different name) in you own module and change it to import all the relevant names from the ast module
in the line processing a Set, replace the generation of a set with a frozenset

You can then use it to safely parse literal sets containing sets. For my Python 3.5, I used:

def frozen_literal_eval(node_or_string):
    """
    Safely evaluate an expression node or a string containing a Python
    expression.  The string or node provided may only consist of the following
    Python literal structures: strings, bytes, numbers, tuples, lists, dicts,
    sets, booleans, and None.

    SPECIAL: This version uses frozensets instead of sets
    """
    # SPECIAL: import names from ast module
    from ast import parse, Expression, Str, Bytes, Num, Tuple, List, Set, Dict
    from ast import NameConstant, UnaryOp, UAdd, USub, BinOp, Add, Sub
    # END SPECIAL
    if isinstance(node_or_string, str):
        node_or_string = parse(node_or_string, mode='eval')
    if isinstance(node_or_string, Expression):
        node_or_string = node_or_string.body
    def _convert(node):
        if isinstance(node, (Str, Bytes)):
            return node.s
        elif isinstance(node, Num):
            return node.n
        elif isinstance(node, Tuple):
            return tuple(map(_convert, node.elts))
        elif isinstance(node, List):
            return list(map(_convert, node.elts))
        elif isinstance(node, Set):
            #SPECIAL: returns a frozenset
            return frozenset(map(_convert, node.elts))
            # END SPECIAL
        elif isinstance(node, Dict):
            return dict((_convert(k), _convert(v)) for k, v
                        in zip(node.keys, node.values))
        elif isinstance(node, NameConstant):
            return node.value
        elif isinstance(node, UnaryOp) and \
             isinstance(node.op, (UAdd, USub)) and \
             isinstance(node.operand, (Num, UnaryOp, BinOp)):
            operand = _convert(node.operand)
            if isinstance(node.op, UAdd):
                return + operand
            else:
                return - operand
        elif isinstance(node, BinOp) and \
             isinstance(node.op, (Add, Sub)) and \
             isinstance(node.right, (Num, UnaryOp, BinOp)) and \
             isinstance(node.left, (Num, UnaryOp, BinOp)):
            left = _convert(node.left)
            right = _convert(node.right)
            if isinstance(node.op, Add):
                return left + right
            else:
                return left - right
        raise ValueError('malformed node or string: ' + repr(node))
    return _convert(node_or_string)

I could the use:

>>> s = '{ 1, 2, {3, 4}}'
>>> frozen_literal_eval(s)
frozenset({1, 2, frozenset({3, 4})})

Thank you! I had to work around a bit more, but it works. The next problem I have are lists in sets. It gives me "TypeError: unhashable type: 'list'", but lists in lists work fine, so why is it a problem now that lists are not hashable either? Any solution for that? — Pidro, Dec 28 '16 at 13:23

score 1 · Answer 2 · edited May 23 '17 at 12:00

Answer to Question 1:

Are sets within sets are possible input?

You may use frozenset to create a nested set object:

Return a new set or frozenset object whose elements are taken from iterable. The elements of a set must be hashable. To represent sets of sets, the inner sets must be frozenset objects. If iterable is not specified, a new empty set is returned.

For example:

>>> frozen_set = frozenset({3, 4})

>>> my_set = {1, 2, frozen_set}
>>> my_set
{frozenset({3, 4}), 2, 1}

Answer to Question 2:

Is there a way to still use literal_eval on {1, 2, {3, 4}}?

No, there is no way as there is no ast.literal_eval for frozenset iterable. There was a proposal in 2008 by Raymond Hettinger for frozenset literal. Guido even pronounced his agreement but then changed his mind. Check this mail for more insights.

But you may use eval instead as:

>>> my_str = '{frozenset({3, 4}), 2, 1}'

>>> eval(my_str)
{1, 2, frozenset({3, 4})}

Before using eval, please also read: Why should exec() and eval() be avoided?

The input is read from a file as string and the file contains a lot of strings. Also they can be nested again and again and again, I would have to edit all of the strings first... I guess that's not very practical. But thank you for your answer! — Pidro, Dec 28 '16 at 07:37

Python: How to use ast.literal_eval() for sets in set?

2 Answers2