13

I am trying to find ways to make better use of my time while programming.

I have a python script that does some heavy work (it can take hours) to finish. Now, most of the work it does is network related, so i have plenty of cpu resources to spare.

If the script was a C binary executable, it would be fine to git checkout onto a different branch and do extra work, I could even modify the binary in disk as it has been copied to ram, so until it finishes running I won't affect program output.

But python scripts are translated, not compiled. What happens if I start tampering with the source file, can i corrupt the programs output, or is the text file and associated imports copied to RAM, allowing me to tamper with the source with no risk of changing the behaviour of the running program?

Makogan
  • 8,208
  • 7
  • 44
  • 112

3 Answers3

18

In general, if you have a single Python file which you run as a script, you're fine. When you run the file, it is compiled into bytecode which is then executed. You can change the original script at this point and nothing breaks.

However, we can deliberately break it by writing some horrible but legal code like this:

horrible.py:

from time import sleep


sleep(10)
import silly
silly.thing()

silly.py:

def thing():
    print("Wow!")

You can run horrible.py and while it is running you can edit silly.py on disk to make it do something else. When silly.py is finally imported, the updated version will be loaded.

A workaround is to put all your imports at the top of the file, which you probably do anyway.

Rob Bricheno
  • 4,467
  • 15
  • 29
5

When a python program is run it is compiled (kinda, more like translated) into a .pyc file that is then run by the python interpreter. When you change a file it should NOT affect the code if it is already running.

Here is a related stackoverflow answer. What will happen if I modify a Python script while it's running?

PeterH
  • 858
  • 1
  • 6
  • 15
  • *All* compilation is a form of translation, unless you think compilation necessarily means "produces machine code for the hardware I am running". – chepner Dec 05 '18 at 19:48
  • Yeah. I lean towards calling it compilation, but I have had people argue that compiling should only be used when talking about compiling to machine code not to byte code used in a interpreter. I am not sure which one is correct so I qualify my statement haha. – PeterH Dec 05 '18 at 21:08
2

Why not have another working directory where you make your modifications? Is there a lot of ancillary data or something that makes it hard to set up a working directory? I.e. if your working directory is A, git clone A B, and then work in B. When you're done, you can pull the changes back from B to A: 

git remote add B ../B
git pull B master
Matthew Woodruff
  • 403
  • 2
  • 8
  • The directory structure is gigantic and the code relies on that specific direcotry structure so i would have to copy zetabytes of memory to have another directory – Makogan Dec 05 '18 at 17:24
  • This might still be doable, although it's not as simple as it would be if the code and data were separate. Suppose you used symbolic links. That is, if you have a directory `A/data`, then in directory `B` type `ln -s /path/to/A/data .` This is assuming that your OS is unixy enough to have symbolic links. – Matthew Woodruff Dec 05 '18 at 17:50
  • I suppose I'm also assuming that your data isn't freely interspersed with the code, so that it's relatively simple to set up your symlinks. – Matthew Woodruff Dec 05 '18 at 17:50