0

I run a Jupyter notebook. One cell contains a control loop of a CPU/memory intensive FEniCS computations. The loop uses print(i) as a progress indicator (where i is the iteration count). As there have been other notebooks running in parallel, swapping happened and the frontend of the notebook has been reloaded. That left me with:

  • the kernel process running (I can see it in the htop output),
  • the kernel is busy (I requested to execute another cell and I am still waiting, the cell is marked [*]),
  • the stdio output of the cell is frozen as it has been somewhen before the reloading (37%),
  • the cell is marked as neither executed nor running ([ ]),
  • the notebook favicon is "busy" (a hourglass),
  • the kernel ignoring (?) interrupt requests - it does not stop when I click the stop button (but I am used to such behaviour when I use FEniCS).

From experience I know, that either:

  • the cell is actually running, the state is there, and I can access it will be accessible after the execution is finished,
  • the cell has hanged (e.g. kernel is deadlocked performing some kind of busy waiting or so, IDK).

I have interrupted the kernel (with either Jupyter notebook frontend or kill -2 - I had tried both). The cell was actually running and the control loop was at 98% (no data lost though). But I still want to know whether there is any way I can get access to stdout stream in cases like this (without interrupting the execution unless it can be resumed exactly where it was interrupted).

abukaj
  • 2,582
  • 1
  • 22
  • 45
  • why are you trying to run another cell while another cell is running? you can't do that. can you instead open 2 jupyter notebooks and run them separately ? – anarchy Jan 08 '23 at 18:53
  • i think you should be using the multiprocessing, multithreading or async or a mix of these so you can do what you need to do and let python decide which cores or threads you need to use. – anarchy Jan 08 '23 at 18:55
  • @anarchy I am not (trying to run another cell while another is running). I "schedule" them to run serially either by "Run all" or more manually. Running multiple notebooks is exactly what I do. "Poor man's parallelism" I'd say. I do not touch multithreading, but FEniCS does. Jupyter does its multiprocessing, Ubuntu - multitasking, and the result is forced reloading of the notebook's frontend, thus lost `stdout` of the cell running in background. – abukaj Jan 16 '23 at 15:32

0 Answers0