Using exec() to make and assign variables from a text file

Question

This question is a follow-up to the question I asked here, which in summary was:

"In python how do I read in parameters from the text file params.txt, creating the variables and assigning them the values that are in the file? The contents of the file are (please ignore the auto syntax highlighting, params.txt is actually a plain text file):

Lx = 512 Ly = 512
g = 400
================ Dissipation =====================
nupower = 8 nu = 0
...[etc]

and I want my python script to read the file so that I have Lx, Ly, g, nupower, nu etc available as variables (not keys in a dictionary) with the appropriate values given in params.txt. By the way I'm a python novice."

With help, I have come up with the following solution that uses exec():

with open('params.txt', 'r') as infile:
    for line in infile:
        splitline = line.strip().split(' ')
        for i, word in enumerate(splitline):
            if word == '=':
                exec(splitline[i-1] + splitline[i] + splitline[i+1])

This works, e.g. print(Lx) returns 512 as expected.

My questions are:

(1) Is this approach safe? Most questions mentioning the exec() function have answers that contain dire warnings about its use, and imply that you shouldn't use it unless you really know what you're doing. As mentioned, I'm a novice so I really don't know what I'm doing, so I want to check that I won't be making problems for myself with this solution. The rest of the script does some basic analysis and plotting using the variables read in from this file, and data from other files.

(2) If I want to wrap up the code above in a function, e.g. read_params(), is it just a matter of changing the last line to exec(splitline[i-1] + splitline[i] + splitline[i+1], globals())? I understand that this causes exec() to make the assignments in the global namespace. What I don't understand is whether this is safe, and if not why not. (See above about being a novice!)

Answer 1

(1) Is this approach safe?

No, it is not safe. If someone can edit/control/replace params.txt, they can craft it in such a way to allow arbitrary code execution on the machine running the script.

It really depends where and who will run your Python script, and whether they can modify params.txt. If it's just a script run directly on a normal computer by a user, then there's not much to worry about, because they already have access to the machine and can do whatever malicious things they want, without having to do it using your Python script.

(2) If I want to wrap up the code above in a function, e.g. read_params(), is it just a matter of changing the last line to exec(splitline[i-1] + splitline[i] + splitline[i+1], globals())?

Correct. It doesn't change the fact you can execute arbitrary code.

Suppose this is params.txt:

Lx = 512 Ly = 512
g = 400
_ = print("""Holy\u0020calamity,\u0020scream\u0020insanity\nAll\u0020you\u0020ever\u0020gonna\u0020be's\nAnother\u0020great\u0020fan\u0020of\u0020me,\u0020break\n""")
_ = exec(f"import\u0020ctypes")
_ = ctypes.windll.user32.MessageBoxW(None,"Releasing\u0020your\u0020uranium\u0020hexaflouride\u0020in\u00203...\u00202...\u00201...","Warning!",0)
================ Dissipation =====================
nupower = 8 nu = 0

And this is your script:

def read_params():
    with open('params.txt', 'r') as infile:
        for line in infile:
            splitline = line.strip().split(' ')
            for i, word in enumerate(splitline):
                if word == '=':
                    exec(splitline[i-1] + splitline[i] + splitline[i+1], globals())

read_params()

As you can see, it has correctly assigned your variables, but it has also called print, imported the ctypes library, and has then presented you with a dialog box letting you know that your little backyard enrichment facility has been thwarted.

As martineau suggested, you can use configparser. You'd have to modify params.txt so there is only one variable per line.

tl;dr: Using exec is unsafe, and not best practice, but that doesn't matter if your Python script will only be run on a normal computer by users you trust. They can already do malicious things, simply by having access to the computer as a normal user.

---

Is there an alternative to configparser?

I'm not sure. With your use-case, I don't think you have much to worry about. Just roll your own.

This is similar to some of the answers in your other question, but is uses literal_eval and updates the globals dictionary so you can directly use the variables as you want to.

params.txt:

Lx = 512 Ly = 512
g = 400
================ Dissipation =====================
nupower = 8 nu = 0
alphapower = -0 alpha = 0
================ Timestepping =========================
SOMEFLAG = 1
SOMEOTHERFLAG = 4
dt = 2e-05
some_dict = {"key":[1,2,3]}
print = "builtins_can't_be_rebound"

Script:

import ast

def read_params():
    '''Reads the params file and updates the globals dict.'''
    _globals = globals()
    reserved = dir(_globals['__builtins__'])
    with open('params.txt', 'r') as infile:
        for line in infile:
            tokens = line.strip().split(' ')
            zipped_tokens = zip(tokens, tokens[1:], tokens[2:])
            for prev_token, curr_token, next_token in zipped_tokens:
                if curr_token == '=' and prev_token not in reserved:
                    #print(prev_token, curr_token, next_token)
                    try:
                        _globals[prev_token] = ast.literal_eval(next_token)
                    except (SyntaxError, ValueError) as e:
                        print(f'Cannot eval "{next_token}". {e}. Continuing...')

read_params()

# We can now use the variables as expected
Lx += Ly
print(Lx, Ly, SOMEFLAG, some_dict)

Output:

1024 512 1 {'key': [1, 2, 3]}