Evaluate a string as a Python literal without try/except

Question

I have in my job many times to parse ASCII headers of files from instruments and most of the time I do that interactively with the IPython console. Usually, I have a header that looks like this:

range = 55
param1 = 0.1
comment = "this is a comment"
parm2  = 0.4
values = [1,2,3,4]

I usually split and take the values on the right, but I want to cast them to the python type that best represent them e.g. [1,2,3,4] I don't want a string but a list. I would like to convert automatically the values to the closest Python type that they can assume (immagine that later I've to add parm1 to parm1 of another file or zip values of two file together), this is a test case:

var = "this should not be displayed!"
testcases = ["55 "," 0.1","[1,2,3]","{1,4,5}","comment comment","'scanning type!'",'{"key": "value"}','""','None','also this is valid',"var"]

I usually use this approach but I have to write a try and except (this for Python3 because of the except):

import ast  
for test in testcases:
       try:
            test = ast.literal_eval(test.strip())
       except(SyntaxError, ValueError) as g:
            test = test.strip() 
       print(test)

Is there any better approach without try and excepts and using the standard Python library or common scientific library (numpy,scipy, pandas) that output the same output of the function I've written? This would speed up a lot my job.

It's impossible to guess a type. E.g. is `"[1,2,3]"` a list or a string with that value? If you give more details about your use case, we'd probably be able to suggest something better. E.g. why do you need to blindly guess a type in the first place? The value of an "ASCII header" is a string! Without knowing what that header stands for, you cannot meaningfully manipulate its value. — ivan_pozdeev, Aug 20 '19 at 17:15
@ivan_pozdeev "[1,2,3]" must be translate as a list, yes, I've used this process so far and I've never encounterd problem. Of course on the left side I've got the metadata I am looking for a way to do it without try and except — G M, Aug 20 '19 at 17:59
@ivan_pozdeev my actual problem is really simple as I've explained: I want to do exactly what I am doing without using try and except, this for me is important for reading header quicker because I am using this solution many time in my job — G M, Aug 20 '19 at 18:05
As a side note, there are two bugs in your code: `test.strip()` doesn't change `test` (should be `test = test.strip()`, and [bare `except:` is an anti-pattern](https://docs.python.org/2/howto/doanddont.html#except) (should be `except SyntaxError:`). — ivan_pozdeev, Aug 20 '19 at 18:13
@ivan_pozdeev thanks for you edit yes actually I should write except(SyntaxError, ValueError) as g but I will lose the compatibility Python3, Python2. My code is not intended as a best practice but a method to evaluate the test case. However I'll delete this question if you think is not appropriate I'll ask in some forum. Thanks for your effort — G M, Aug 21 '19 at 08:37

ivan_pozdeev · Answer 1 · 2019-08-20T18:23:36.287

2

As per Cheap exception handling in Python?, exception handling is designed to be cheap and a part of normal workflow in Python. This is the reason why in Python, there's no need for TryParse-like methods.

So there's nothing wrong with using try/except, and ast.literal_eval is the fastest method available in Python to evaluate a Python literal.

edited Aug 20 '19 at 18:23

answered Aug 20 '19 at 18:17

ivan_pozdeev

33,874
19
107
152

Indeed, totally agree with you, even if my answer does a dirty trick to avoid the try/catchs , I think is cleaner and even clear for the code's readers to understand. In my answer the two examples is, IMO more clear the try/catch example, I will up vote this answer. – developer_hatch Aug 20 '19 at 21:33
I know theres nothing wrong but in my rutine doing it often in the shell is slow to write, I'm just asking if there is a method without try and except that works for my test cases. I think the question is very clear. – G M Aug 21 '19 at 08:39
@GM None that I know of. And I doubt there can be a universal one because what value would it return to indicate an error? – ivan_pozdeev Aug 21 '19 at 16:17

developer_hatch · Answer 2 · 2019-08-20T21:34:39.617

Edit:

Using Success and Failure can do the trick, no Try/catch:

from returns.functions import safe
from returns.result import Result, Success, Failure

@safe
def tryParse(v):
  eval(v)

testcases = ["55 ",
" 0.1",
"[1,2,3]",
"{1,4,5}",
"INVALID VALUE",
"'scanning type!'",
'{"key": "value"}',
'""',
'None',
"OTHER INVALID VALUE",
'"VALID VALUE"']

res = [tryParse(x).unwrap() for x in testcases if type(tryParse(x)).__name__ == '_Success']

print(res)

I still think that the simpler way is to use exception handling. I think is cleaner and even clear for the code's readers to understand:

testcases = ["55 ",
" 0.1",
"[1,2,3]",
"{1,4,5}",
"INVALID VALUE",
"'scanning type!'",
'{"key": "value"}',
'""',
'None',
"OTHER INVALID VALUE",
'"VALID VALUE"']


INVALID = "NONE_VALID_VALUE"

def tryParse(v):
  try:
    return eval(v)
  except Exception:
    return INVALID


res = [tryParse(x) for x in testcases if tryParse(x) != INVALID]

print(res)

OUTPUT:

[55, 0.1, [1, 2, 3], {1, 4, 5}, 'scanning type!', {'key': 'value'}, '', None, 'VALID VALUE']

Thanks but I am looking for a solution without try and except — G M, Aug 20 '19 at 18:01
Thanks a lot for your efforts, I used eval in the past but this may be dangerous I've add a test case for showing this behaviour. — G M, Aug 21 '19 at 08:34

score -2 · Answer 3 · answered Aug 20 '19 at 17:32

Not exactly sure how you would go about this, but here's something interesting I found out when trying to find a solution for you:

for i in testcases:
    try:
        print(eval(f"type({i})", {}))

It doesn't return strings, but maybe you can work around it with try/except cases. This is also not at all secure - if this is for anything you plan to release. The user can escape this sequence by having an input of "){malicious code here}". This is literally just running the string through an evaluator, that will evaluate based on executing the string as python code. The blank dict passed through will prevent anyone from using any external libraries.

Thanks for you effort but in my question I've ask for a solution without try and except — G M, Aug 20 '19 at 18:07

Evaluate a string as a Python literal without try/except

3 Answers3