1

I am working with a set of strings that could contain math operators, either '+' or '-' for now. Initially i used eval which was pretty straightforward. I am trying to move away from eval due to its known issues but mostly am trying to improve the speed. Here is a typical set of string that i would work on :

DAC = ('12','0x1E','34','12+20','2+0x1F')

Taking each value and applying the eval will give the results i need. To move away from this I tried a few methods, they work fine until i hit into a math operator. I did read about AST module and am gonna work on that but would like to get some feedback if its the right track to go in terms of improving the speed or any other suggestions the community can give me..

Thanks

user2345778
  • 31
  • 1
  • 6

2 Answers2

3

What is wrong with literal_eval? That's probably your best bet short of writing or finding a lex parser, which is certainly overkill for this situation.

>>> DAC = ('12','0x1E','34','12+20','2+0x1F')
>>> from ast import literal_eval

You could use a list comprehension

>>> [literal_eval(i) for i in DAC]
[12, 30, 34, 32, 33]

Or map

>>> list(map(literal_eval, DAC))
[12, 30, 34, 32, 33]
Cory Kramer
  • 114,268
  • 16
  • 167
  • 218
1

As I measured, in Python 3 where it's able to use operators, ast.literal_eval is a bit slower than plain eval. So the fastest approach might be:

def myeval(s):
    try: return int(s, 0)
    except ValueError: return eval(s)

Having placed this and your definition of DAC in x.py, I then measure:

aleax-macbookair4:~ aleax$ python3 -mtimeit -s'import x' '[eval(a) for a in x.DAC]'
10000 loops, best of 3: 34 usec per loop
aleax-macbookair4:~ aleax$ python3 -mtimeit -s'import x' '[x.myeval(a) for a in x.DAC]'
10000 loops, best of 3: 22.9 usec per loop

so, a modest speedup.

Actually, for once, a "LBYL" approach may be a little bit better than the usually preferable "EAFP" one: adding to x.py the following

def alteval(s):
    if '+' in s or '-' in s: return eval(s)
    else: return int(s, 0)

I then measure:

aleax-macbookair4:~ aleax$ python3 -mtimeit -s'import x' '[x.alteval(a) for a in x.DAC]'
100000 loops, best of 3: 18.1 usec per loop

for another, though smaller, increase in performance. (I've also tried using re to find out if s has a plus or minus, but that appears to be slower).

Alex Martelli
  • 854,459
  • 170
  • 1,222
  • 1,395
  • Thks Alex, so basically i only use eval for those with math opertors and that should get me the increase in performance – user2345778 Jan 02 '15 at 22:17
  • Right, that the core idea. In Python 3, using `ast.literal_eval` is safer, but it can cost you a modicum of performance. – Alex Martelli Jan 02 '15 at 22:18