OSError : [Errno 12] Not enough Space on os.execl call

Question

So I was messing around with a script that is supposed to restart itself using os.execl. It is working a few times but after ~ 30 or 40 calls it crashes;

Traceback (most recent call last):
  File "C:\Users\Admin#\PycharmProjects\DiscordBot_Fred_the_Financier\test_suit.py", line 9, in <module>
    os.execl(sys.executable, sys.executable, *(*sys.argv, code))
  File "C:\Users\Admin#\AppData\Local\Programs\Python\Python37\lib\os.py", line 540, in execl
    execv(file, args)
OSError: [Errno 12] Not enough space

So this is the whole code I'm running actually:

import sys 
import os

print(sys.argv) # print args
code = "" # placeholder for mutable args
os.execl(sys.executable, sys.executable, *(*sys.argv, code)) # passing new args and replacing process

I have literally no idea why and how this error occurs.
All my drives have >200 GB free storage and my RAM more than 17 GB as well.
I'm running this code via terminal on Win10 64bit python 3.7.
Thank you very much for your help!

P.S. I apologize if there is already an answer to this problem but I could not find one.

Answer 1

Are you opening any huge files in your script? Most likely you are not closing those file handles and they keep accumulating. Once the script crashes all the handles will be released and you see the 200GB

While running the script, can you keep an eye on disk usage? Do you see it rising continuously? (at least after the subsequent calls of exec)

EDIT: I see now that the question was asked in 'Mar 28 '20 at 18:58'. I do not know why I saw it in main list. Question can be closed if OP does not reply or gives more info.

Answer 2

I haven't found why this error happens, but I am assuming it has something to do with the parent process not being able to fully close (maybe due to references left or something similar). I actually found a workaround which I am leaving here in case it helps you or someone else with the same problem.

The workaround is using subprocess.Popen instead of os.execl, and immediately adding os._exit(1). What the latter does is immediately shut down the parent process and thus free all resources. So if you do this, you won't get the 'Not enough space' error, no matter how many times you restart the process. To verify this, I made a test, by running these two lines of code in an infinite loop (although in retrospective, it's not necessary as a second iteration is impossible due to os._exit(1) statement), and leaving it running for a while, and writing to a .txt file per each process replacement. After some considerable amount of time, I stopped the final process by interrupting it with Ctrl+C, and when I opened the file, I found the value 15127, which means that the process had been replaced 15127 times before I interrupted it and I never had any exceptions raised due to space issues.

Additionally, as with os.execl, you can pass command-line arguments by passing sys.argv, for example by typing:

import os
import sys
import subprocess

print(sys.argv)
code = ""
subprocess.Popen([sys.executable, *(*sys.argv, code)])
os._exit(1)

I omitted the second sys.executable because I am not sure of what you are trying to do and if it would work in subprocess.Popen, but given the similarity of these two commands, I'd say it would work the same way if you typed subprocess.Popen(sys.executable, sys.executable, *(*sys.argv, code) instead.

EDIT: Actually, with subprocess.Popen, you need to pass all arguments as a list, as otherwise the second argument you pass will be considered a bufsize according to the method signature (see https://docs.python.org/3/library/subprocess.html#popen-constructor for further details), or as a string concatenation, if possible (for example, subprocess.Popen(sys.executable + " " + code). Also, I added a missing parenthesis.