-1

I have a python script that runs without any problem when I run it via command line or through an editor. However, once I set it up to run automatically via Task Scheduler, I am getting MemoryErrors in my logs.

This is the error message:

MemoryError: Unable to allocate 2.03 MiB for an array with shape (10252, 52) and data type object

enter image description here

TylerH
  • 20,799
  • 66
  • 75
  • 101
Daniel Pelnar
  • 1
  • 1
  • did u set 1 or 2 seconds time.sleep? –  Sep 08 '20 at 09:45
  • I will try. Thanks. Could you elaborate on why pausing the script for a second or two might solve the problem? @CYREX – Daniel Pelnar Sep 08 '20 at 11:26
  • https://stackoverflow.com/a/34876536/13824946 –  Sep 08 '20 at 12:12
  • @CYREX If I understand it correctly, the thread you sent me is describing a situation when time.sleep() is actually causing memory leaks. I am not sure how it can prevent them. Any idea? – Daniel Pelnar Sep 08 '20 at 14:09
  • basing on your error message you have to deal with threads and time.sleep because launching the entire job results in memory leak or no memory at all. so you have to read by 1000 lines for each thread, once the first thread finished reading 1000 lines, it will be slept or process killed then the second thread start to do the job and so on. this is what i meant. –  Sep 08 '20 at 14:18
  • I have tried time.sleep(arg) and gc.collect() in vain. Does anyone have other tips what might be wrong? Other scripts run well via Task Scheduler. It is just this one. @CYREX – Daniel Pelnar Sep 10 '20 at 07:30
  • https://www.google.com/amp/s/www.geeksforgeeks.org/python-different-ways-to-kill-a-thread/amp/ –  Sep 10 '20 at 09:08

1 Answers1

0

I have not solved my question directly but still I have a solution that might be handy for some people.

At the end, I gave up on Task Schedule and instead created a new Python script which runs all my Python scripts at a specific time. The library is called Schedule.

import schedule
from MODULE_1 import FUNCTION_1  
from MODULE_2 import FUNCTION_2 

schedule.every().monday.at(13:20).do(FUNCTION_1)
schedule.every().monday.at(20:10).do(FUNCTION_2) 

while True:
    try:
        schedule.run_pending()
    except Exception as f: 
        e = datetime.datetime.now()
        print(e.strftime("%Y-%m-%d %H:%M:%S"))
        print("ERROR TYPE:")
        print(f) 
    time.sleep(1)
Daniel Pelnar
  • 1
  • 1