How to (intermittently) skip certain cells when running IPython notebook?

Question

I usually have to rerun (most parts of) a notebook when reopen it, in order to get access to previously defined variables and go on working.

However, sometimes I'd like to skip some of the cells, which have no influence to subsequent cells (e.g., they might comprise a branch of analysis that is finished) and could take very long time to run. These cells can be scattered throughout the notebook, so that something like "Run All Below" won't help much.

Is there a way to achieve this?

Ideally, those cells could be tagged with some special flags, so that they could be "Run" manually, but would be skipped when "Run All".

EDIT

%%cache (ipycache extension) as suggested by @Jakob solves the problem to some extent.

Actually, I don't even need to load any variables (which can be large but unnecessary for following cells) when re-run, only the stored output matters as analyzing results.

As a work-around, put %%cache folder/unique_identifier to the beginning of the cell. The code will be executed only once and no variables will be loaded when re-run unless you delete the unique_identifier file.

Unfortunately, all the output results are lost when re-run with %%cache...

EDIT II (Oct 14, 2013)

The master version of ipython+ipycache now pickles (and re-displays) the codecell output as well.

For rich display outputs including Latex, HTML(pandas DataFrame output), remember to use IPython's display() method, e.g., display(Latex(r'$\alpha_1$'))

If you don't need to redisplay the output, you can define your own skip magic like I did here: http://stackoverflow.com/a/43584169/4098821 — Robbe, Apr 24 '17 at 09:21

score 70 · Answer 1 · edited Nov 21 '21 at 22:16

70

Though this isn't exactly what you seem to be looking for, if you wish to entirely omit the execution of a cell (where no cached results are loaded), you can add the following hack at the beginning of a cell (assuming you are using a Unix-based OS):

%%script false

or a variant (working as of early 2020 -- see here for explanation):

%%script false --no-raise-error

edited Nov 21 '21 at 22:16

wjandrea

28,235
9
60
81

answered May 18 '16 at 19:09

Mark

3,357
30
31

1

Anyone know how to do this on Windows? – royco Feb 18 '17 at 22:10
4

It works perfectly on windows. The notebook output will be "Couldn't find program: 'false'". – DJV Jun 13 '19 at 07:33
2

This stopped working some time in 2019. You can find new workarounds [here](https://stackoverflow.com/a/59806154/9201239). – stason Jan 19 '20 at 00:40
As a heads up, this gets rid of existing output. Not useful if you want to keep previous output of the cell. – Suhas Apr 15 '20 at 23:07
Here's another similar one (`%%script echo`): https://stackoverflow.com/a/61544643/1090455 – K3---rnc Jul 13 '20 at 14:33
1

instead use this as it works on all known platforms: %%script echo skipping – dabru Sep 17 '20 at 17:09
2

What about `%%script true` instead of `%%script false --no-raise-error`? – wjandrea Nov 21 '21 at 19:37
I would put the option first to show that it belongs to the `%%script` magic, not the `false` command: `%%script --no-raise-error false` --- @wjandrea `%%script true` will probably fail on some non-POSIX platforms like Windows. Also the falsy return status could be probably used as an explicit indication that the cell did not (correctly) execute? – pabouk - Ukraine stay strong Jun 04 '22 at 13:15

score 42 · Accepted Answer · edited Nov 21 '21 at 19:30

42

Currently, there is no such feature included in the IPython notebook. Nevertheless, there are some possibilities to make your life easier, like:

use the %store or maybe better the %%cache magic (extension) to store the results of these intermittently cells, so they don't have to be recomputed (see https://github.com/rossant/ipycache)
add a if==0: before the cells you don't want to execute
convert these cells to raw cells (but you will loose the already stored output!)

(see discussion at https://github.com/ipython/ipython/issues/2125)

edited Nov 21 '21 at 19:30

wjandrea

28,235
9
60
81

answered Oct 11 '13 at 12:41

Jakob

19,815
6
75
94

1

Great to know about ipycache extension, which functions similarly to what I'm doing (manually) now, but only better! The `%%cache` magic effectively serves as the special flag when "Run All", and manual re-run requires simply cache file deletion. Brilliant! Thanks, Jakob~ – herrlich10 Oct 11 '13 at 15:41
Indeed using %%cache is much more convenient than manually handling with %store. %store need to send one by one the variables to save, but can read all at time, little pointless. %%cache works perfect for whole cell, and %store works better for single variables. – m3nda Apr 11 '18 at 03:26
`ipycache` is no longer maintained- do you know any other solutions? – Chris_Rands Feb 04 '20 at 13:18
@Chris_Rands currently I use the freeze extension. You might ask Rossant directly. – Jakob Feb 04 '20 at 20:21

score 20 · Answer 3 · edited Nov 21 '21 at 22:13

20

Here's a simple and universal solution without the need for workarounds: Simply type this as the top line of the cell to skip the cell:

%%script echo skipping

It's tested on Windows and Mac with recent Jupyter, and I think it should work on other Unix-like platforms as well because they also have an echo command. Some of the other proposed solutions are more platform-specific.

Of course you can put what ever text you like instead of "skipping". When you execute the cell, it will merely print this text instead of executing the code in the cell.

edited Nov 21 '21 at 22:13

wjandrea

28,235
9
60
81

answered May 01 '20 at 14:41

dabru

786
8
8

Works in Hydrogen on Ubuntu – wjandrea Nov 21 '21 at 22:18
Works much better than `%% false` as it doesn't create an error. This is very good if you don't need to keep old output. – Olsgaard Oct 19 '22 at 06:37
Actually it turns out this works but does normally create an (ignored) error on Windows unless you have an echo command in your path (e.g. from installing cygwin, gnuwin32, bash, etc.) – dabru Oct 19 '22 at 15:51
Works beautifully on Google Colab too... – Reuven Jan 31 '23 at 05:16

score 11 · Answer 4 · answered Dec 15 '19 at 02:55

11

If no cached results are expected to be loaded, I find the Freeze nbextension quite useful to this end.

Although unofficial, I strongly recommend to give these notebook extensions a try if you have never used them before.

To install the extension machinery,

$ pip install jupyter_contrib_nbextensions && jupyter contrib nbextension install

To enable the Freeze extension, launch jupyter notebook and open a new notebook, from the menu select Edit > nbextensions config, and then check Freeze.

answered Dec 15 '19 at 02:55

herrlich10

6,212
5
31
35

3

Looks like the best solution. When hitting "Restart and Run All" or other commands, the frozen cells won't be re-executed but the cell outputs are still there. – shaoyl85 Apr 27 '20 at 21:10
2

This is the best solution. The cell won't run, even you press run and the output will still be there. – Bengi Koseoglu Dec 09 '20 at 15:28

score 2 · Answer 5 · answered May 18 '20 at 08:03

This question is a bit older, but the most convenient answer seems to be missing. You can use the 'initialization cells' from the NBextensions. Once installed/activated, you can in any notebook mark cells as 'initialization cells' which can then be run with a specific button.

Install NBextensions:here
Activate 'initialization cells' when you launch the jupyter dashboard
In your notebook, in the 'view' menu choose 'cell toolbar' then 'initialization cell'
Next to each cell there is now a checkbox. Mark all the cells you want to run for initialization
When reopening a notebook, click the button that looks like a pocket calculator to run all the initialization cells.

score 2 · Answer 6 · answered Nov 23 '20 at 13:31

2

The simplest way to skip python code in jupyter notebook cell from running, I temporarily convert those cells to markdown.

answered Nov 23 '20 at 13:31

Rajan Verma - Aarvy

1,969
18
20

4

but in this way you lose the output – Nisba Jan 29 '21 at 15:17
why someone needs output for the code he/she doesn't want to run? – Rajan Verma - Aarvy Jan 29 '21 at 20:31
3

In data science you don't want to run the whole notebook every time, but still you want to be able to see the results – Nisba Jan 30 '21 at 18:37

score 1 · Answer 7 · edited Nov 21 '21 at 20:19

The %%script false solution stopped working some time in 2019.

Here are some other available workarounds. These are based on programs ignoring their arguments when you tell them not to expect any. Here are some easy examples:

Perl:

%%perl -e0

for i in range(10): print(i)

Here you're running: perl -e '0' cellcontents

A more memorable version:

%%perl -eat

for i in range(10): print(i)

Here you're running: perl -e 'at' cellcontents

Bash:

%%bash -c :

for i in range(10): print(i)

: is a no-op in bash, so you're running: bash -c : cellcontents

I haven't looked at the external magic implementation code, but I'm pretty sure "cellcontents" are passed as arguments and won't be interpreted by shell by mistake, say if you were to include ; in them and accidentally inject some bad code. But I can't guarantee you that.

I'm sure you can come up with other creative solutions, by looking at the supported programs here: https://ipython.readthedocs.io/en/stable/interactive/magics.html#cell-magics

%%script false --no-raise-error works for the purpose on the Jupyter I tried today — Davide Fiocco, Jan 19 '20 at 21:09
indeed, it does, but it's a way too long to type and remember... Ideally, ipython should have a `%%script noop` or something similar, so it's a feature and not a workaround. — stason, Jan 19 '20 at 22:34
Why such complicated solutions. There is a POSIX command which always returns true. You can use: `%%script true` — pabouk - Ukraine stay strong, Jun 04 '22 at 12:24

score 0 · Answer 8 · answered Sep 08 '22 at 04:31

For the cells you wish to skip when you press Run All you can use try/except blocks, where you try to display the already calculated information in the try block, and perform the calculation in the except block.

Take this cell for example:

my_results = 2 # Assume this is a bigger calculation
print(my_results)
print('displayed my results.')

To skip the calculation for this cell in subsequent runs, change the contents of this cell to the following:

try:
  print(my_results)
  print('displayed state value')
except:
  my_results = 2 # Assume this is a bigger calculation
  print(my_results)
  print('displayed newly calculated value')

The first time you run this cell, it will attempt to print the value of the state variable my_results. This throws an exception so it jumps to the except block and does the actual calculation of my_results (which in this case just makes it equate to 2). At the end of the first run the output for this cell would be:

2
displayed newly calculated value

When you run the cell a second time (whether that be manually or via Run All), the try block will first execute but this time since the variable is available in the state, it does not throw an exception. Instead it displays the result and the except block is never run. At the end of the second run the output of this cell would be:

2
displayed state value

This doesn't meet your explicit criteria that the cell should be completely skipped, but effectively the calculation is skipped.

If the displaying of the information is more complex than using a single print or display, it would probably be cleaner if you make the displaying routine into a function like so:

def print_my_results(my_result):
  print(my_result)
  print('displayed my result.')

try:
  print_my_results(my_results)
except:
  my_results = 2 # Assume this is a bigger calculation
  print_my_results(my_results)

this is clever, but can you achieve the same thing with a simple if...else block? — Matt Montag, Apr 16 '23 at 20:05
Good observation @MattMontag, I had to think about it for a little bit but I would say this try/catch is better than an if/else block: With an if/else you need to track and account for every used variable in the if statement. There can be a situation where you forget to update that if statement. With a try/catch you don't need to worry about that. You can just focus on printing the variable and if you forget a calculation it will throw another exception in the `except` block prompting you to add the missing calculation upstream or in the `except` block. — Daniel, Apr 21 '23 at 02:54

score 0 · Answer 9 · answered May 14 '23 at 17:19

I'm using most simple solution. Just add % as first character of the cell and this cell will be skipped. As cons: Error message (one line under the cell) will be shown, but it don't inflict any troubles to running other cells. As pros: It's quick on/off for specific cells.

score 0 · Answer 10 · answered Jun 15 '23 at 10:10

0

A workaround I sometimes use:

if 0:
    time_consuming_function()

answered Jun 15 '23 at 10:10

Yoda

574
1
9
21

How to (intermittently) skip certain cells when running IPython notebook?

10 Answers10

Linked

Related