0

There is a very nice answer on how to pass extra arguments to derivative evaluator in SciPy's solve_ivp function using a lambda expression. This approach is official way to deal with derivatives depending on extra arguments as indicated on SciPy's github issue tracker.

This approach seems to be working really well while run inside of Python script, however once run inside of Jupyter environment's Python kernel it gets some unpredictable behavior for which I fail to identify the cause.

Following example is based on an official example.

from scipy.integrate import solve_ivp
import numpy as np

def exponential_decay(t, y, alpha, beta): return -alpha*y + beta

for alpha in np.linspace(0.5, 0.7, 5) :
    beta = 1.
    sol = solve_ivp(lambda t, y: exponential_decay(t, y, alpha, beta), [0, 10], [2, 4, 8])

Once it is put inside of Jupyter notebook the local variable of scope inside of the for loop, alpha is causing a NameError. Once run again without the loop it is working well. I do restart the kernel before each time I run all cells to make sure environment has no defined globals. This error persists with both Python 2 and Python 3 kernels.

The details can be find in this public gist I prepared. The stack trace, as requested from the comments is the following:

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<timed exec> in <module>

~/.pyenv/versions/3.6.7/envs/jupyter3/lib/python3.6/site-packages/scipy/integrate/_ivp/ivp.py in solve_ivp(fun, t_span, y0, method, t_eval, dense_output, events, vectorized, **options)
    454         method = METHODS[method]
    455 
--> 456     solver = method(fun, t0, y0, tf, vectorized=vectorized, **options)
    457 
    458     if t_eval is None:

~/.pyenv/versions/3.6.7/envs/jupyter3/lib/python3.6/site-packages/scipy/integrate/_ivp/rk.py in __init__(self, fun, t0, y0, t_bound, max_step, rtol, atol, vectorized, first_step, **extraneous)
     98         self.max_step = validate_max_step(max_step)
     99         self.rtol, self.atol = validate_tol(rtol, atol, self.n)
--> 100         self.f = self.fun(self.t, self.y)
    101         if first_step is None:
    102             self.h_abs = select_initial_step(

~/.pyenv/versions/3.6.7/envs/jupyter3/lib/python3.6/site-packages/scipy/integrate/_ivp/base.py in fun(t, y)
    137         def fun(t, y):
    138             self.nfev += 1
--> 139             return self.fun_single(t, y)
    140 
    141         self.fun = fun

~/.pyenv/versions/3.6.7/envs/jupyter3/lib/python3.6/site-packages/scipy/integrate/_ivp/base.py in fun_wrapped(t, y)
     19 
     20     def fun_wrapped(t, y):
---> 21         return np.asarray(fun(t, y), dtype=dtype)
     22 
     23     return fun_wrapped, y0

<timed exec> in <lambda>(t, y)

NameError: name 'alpha' is not defined

I'm not very experienced with Jupyter and I am out of ideas, so I'd like to ask you for some advices, here are some basic questions I have

  1. What is a possible thing I could check further? I did test for name errors inside of the loop using try ... catch NameError : ... approach and everything was well defined as expected, problem occurs once inside of the lambda function.
  2. What are best practices to work with Jupyter notebooks to avoid problems with precomputed values of variable and their scope? I like writing simple scripts as they allow me to not worry about such things, however being able to comment sections of code with LaTeX is a huge advantage Jupyter brings which I also value a lot.
  3. I've been trying to research this problem a bit and maybe it isn't really the scope problem but more of a pointer / reference kind of problem (based on this answer), however if that is the case, why the problem occurs only in Jupyter notebook and not in simple Python script?

Any help, advices, examples will be appreciated. Thanks!

Marek
  • 1,413
  • 2
  • 20
  • 36
  • 1
    Can you please post the entire stack trace of the `NameError`? – Zionsof May 01 '19 at 07:51
  • I gotta say that at my end it works fine, I get the results and no name error whatsoever, from both methods (with and without loop). I'm running python2 though in my kernel, and I can't install python3 right now though, so that might be it. Can you try running with python2? – Zionsof May 01 '19 at 08:05
  • @zionsof the script works fine for me as well, problem is when I run it inside of Jupyter, and even there sometimes it works sometimes doesn't, if happens to work at first time I have to add an extra variable set to a constant to reproduce the error. – Marek May 01 '19 at 08:09
  • 1
    I know, same here. I meant in jupyter with kernel of python2 it works for me. I asked if you can try running the jupyter with kernel of python2 instead of 3? – Zionsof May 01 '19 at 08:27
  • @zionsof as requested I just changed kernel to Python 2 and re-run all the cells, error still occurs. – Marek May 02 '19 at 03:01
  • That's strange. It looks like something is causing the `%%time` magic to run your code with different global and local namespaces. That would prevent the `lambda` from seeing `alpha` or any other variables created by the timed code. It doesn't explain why you'd have those separate namespaces, though. – user2357112 May 02 '19 at 03:14
  • I can't reproduce this error in either IPython or a Jupyter notebook. What exact IPython and Jupyter versions are you using? – user2357112 May 02 '19 at 03:14
  • I suspect you've got some sort of weird configuration causing this problem. What happens if you run `print(locals() is globals())` in a cell by itself? It should normally print `True`. – user2357112 May 02 '19 at 03:23
  • @user2357112 `print(locals() is globals())` gives `True` and `IPython` version is ` 7.4.0` (not sure if it's same as Jupyter version...), I commited those to gist repository, let me know if that information is sufficient or anything else I can check. – Marek May 02 '19 at 04:37
  • That eliminates some possible categories of problems. What happens if you run `import IPython; IPython.get_ipython().user_ns is globals()`? Also try running `foo = 'foo'; (lambda: foo)()` under `%%time`. – user2357112 May 02 '19 at 05:57

0 Answers0