How to run a large amount of python code from a string?

Question

I need to be able to run a large amount of python code from a string. Simply using exec doesn't seem to work, as, while the code runs perfectly in a normal setting, doing it this way seems to throw an error. I also don't think I can just import it as it it hosted on the internet. Here is the code:

import urllib.request

URL = "https://dl.dropboxusercontent.com/u/127476718/instructions.txt"  

def main():
    instructions = urllib.request.urlopen(URL)

    exec(instructions.read().decode())

if __name__ == "__main__":
    main()

This is the error I've been getting:

Traceback (most recent call last):
  File "C:\Python33\rc.py", line 12, in <module>
    main()
  File "C:\Python33\rc.py", line 9, in main
    exec(instructions.read().decode())
  File "<string>", line 144, in <module>
  File "<string>", line 120, in main
NameError: global name 'Player' is not defined

The code I'm trying to run is available in the link in the first code snippet.

If you have any questions I'll answer them. Thank you.

You might want to give some thought to the security implications of this design. — NPE, Apr 14 '13 at 09:01
On NPE's point, it may be useful to consider PyPy's [sandboxing](http://doc.pypy.org/en/latest/sandbox.html) feature. — Henry Gomersall, Apr 14 '13 at 09:09
Adding on to NPE's point, note this from the urllib docs: `When opening HTTPS URLs, it does not attempt to validate the server certificate. Use at your own risk!` In other words, man-in-the-middle attacks are possible, and this design would allow an attacker to execute arbitrary code on your system. — Mark Amery, Apr 14 '13 at 09:11
Is it still an issue given that the way I intend to use this is to write the code it links to myself, host it on dropbox and give this code out as an executable. (Thus making updating the code much easier). — user1149589, Apr 14 '13 at 09:19

Eryk Sun · Accepted Answer · 2013-04-14T14:20:32.220

Without specifying globals, the exec function (Python/bltinmodule.c) uses PyEval_GetGlobals() and PyEval_GetLocals(). For the execution frame of a function, the latter creates a new f_locals dict, which will be the target for the IMPORT_NAME, STORE_NAME, LOAD_NAME ops in the compiled code.

At the module level in Python the normal state of affairs is globals() == locals(). In that case STORE_NAME is using the module's globals, which is what a function defined within the module will use as its global namespace. However, using separate dicts for globals and locals obviously breaks that assumption.

The solution is to to manually supply globals, which exec will also use as locals:

def main():
    instructions = urllib.request.urlopen(URL)
    exec(instructions.read().decode(), globals())

You could also use a new dict that has __name__ defined:

def main():
    instructions = urllib.request.urlopen(URL)
    g = {'__name__': '__main__'}
    exec(instructions.read().decode(), g)

I see in the source that the current directory will need a sound file named "pickup.wav", else you'll just get another error.

Of course, the comments about the security problems with using exec like this still apply. I'm only addressing the namespace technicality.

score 1 · Answer 2 · edited May 23 '17 at 11:57

First I thought you might try __import__ with a StringIO object. Might look something like StackOverflow: Local Import Statements in Python.

... but that's not right.

Then I thought of using the imp module but that doesn't seen to work either.

Then I looked at: Alex Martelli's answer to Use of Eval in Python --- and tried to use it on a silly piece of code myself.

I can get the ast object, and the results of the compile() from that (though it also seems that one can simply call compile(some_string_containing_python_source, 'SomeName', 'exec') without going through the ast.parse() intermediary step if you like. From what I gather you'd use ast if you wanted to then traverse the resulting syntax tree, inspecting and possibly modifying nodes, before you compiled it.

At the end it seems that you'll need to exec() the results of your compile() before you have resulting functions, classes or variables defined in your execution namespace.

`__import__` takes module name as argument. The question you link to uses `StringIO` as an example of a module to import. — Janne Karila, Apr 14 '13 at 09:48
Yep, I see that now; hoping to fix that soon (using the `imp` module). — Jim Dennis, Apr 14 '13 at 09:51

score 0 · Answer 3 · edited Apr 14 '13 at 09:06

0

You can use pipe to put all strings into a child process of python and get output result from it.

Google os.popen or subprocess.Popen

edited Apr 14 '13 at 09:06

jamylak

128,818
30
231
230

answered Apr 14 '13 at 09:03

richselian

731
4
18

This just seems to throw a different error `FileNotFoundError: [WinError 2] The system cannot find the file specified` – user1149589 Apr 14 '13 at 09:20
You don't have to put code strings into a file. you just need to start a python sub-process and write code strings to its pipe.stdin. – richselian Apr 14 '13 at 12:17

How to run a large amount of python code from a string?

3 Answers3