Launching a python script as a background job on Jupyter

Question

I am trying to run a *.py file as a background service in Jupiter notebook.

from IPython.lib import backgroundjobs as bg
jobs = bg.BackgroundJobManager()
jobs.new(%run -i "script.py") # Not working
jobs.new("script.py") # Not working

Answer 1

Ipython/Jupyter background jobs are designed to run either plain code to eval (string), or function. Files and ipython magic commands are not supported.

One thing you can do is to simply read file content and pass it to eval:

from IPython.lib.backgroundjobs import BackgroundJobFunc

with open('script.py') as code:
    job = BackgroundJobFunc(exec, code.read())

result = job.run()

BackgroundJobManager is pretty much the same, but a little bit "smarter".

Side note: all background machinery behind this interfaces runs in threads of the same process and share interpreter state and output. So, just keep in mind:

• this is not suited for computational-heavy scripts
• never run untrusted code this way — this applies to eval overall, but in this case you can into situation when you'll never get GIL back to your "frontend" thread
• avoid scripts that use stdout, most probably those will clutch with your main thread