53

If I have several IPython notebooks running on the same server. Is there any way to share data between them? For example, importing a variable from another notebook? Thanks!

Kyle Siegel
  • 733
  • 1
  • 7
  • 8
  • The code running each notebook is its own process, so they can't share variables. You'll need to serialise them in some form or another to transfer them between notebooks. The best way to do this depends on your data, but CSV or JSON are two popular formats. 'Pickle' can save most Python variables, but if you also want to archive your data, it's not recommended, because it depends on the specific Python version you're using. – Thomas K Jul 27 '15 at 19:06
  • This facility sounds similar to what I'd like to do. I'd like to keep some data that is "persistent" between Kernel resets. I suspected JSONing data out to a file and rereading it at the beginning of the notebook would be the easiest. – RufusVS Mar 17 '18 at 00:24
  • Can use parquet for long-term storage and preserving datatypes https://stackoverflow.com/a/73500091/5957834 – Nandesh Aug 26 '22 at 11:09
  • Does this answer your question? [Share variables between different jupyter notebooks](https://stackoverflow.com/questions/35935670/share-variables-between-different-jupyter-notebooks) – YaOzI Dec 01 '22 at 06:34

5 Answers5

67

This works for me :

The %store command lets you pass variables between two different notebooks.

data = 'this is the string I want to pass to different notebook' %store data

Now, in a new notebook… %store -r data print(data) this is the string I want to pass to different notebook

I've successfully tested with sklearn dataset : 

from sklearn import datasets

dataset = datasets.load_iris()

%store dataset

in notebook to read data :

%store -r dataset

src : https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/

blue-sky
  • 51,962
  • 152
  • 427
  • 752
  • Related but with some other features that folks may find handy, there is [IPython data-vault](https://github.com/krassowski/data-vault) described as, "IPython magic for simple, organized, compressed and encrypted: storage & transfer of files between notebooks." – Wayne Jan 13 '20 at 16:05
17

Notebooks in Jupyter Lab can share the same kernel. In your notebook you can choose the kernel of another notebook and variables from the other notebook will be available in both notebooks.

screenshot Jupyter Lab kernel selection popup

  1. Click on the button that describes your current kernel.
  2. Choose the kernel of the other notebook, whose variables you want to access.
Stefan_EOX
  • 1,279
  • 1
  • 16
  • 35
7

IPython supports the %store magic (here is the documentation). It seems to have the same constraints of pickle: if the file can be pickled it will also be storable.

Anyway, it will work for sure with common Python types. Here's a basic example:

var_1 = [1,2,3,4] #list
var_2 = {'a':1,'b':2,'c':3} #dict
var_3 = (6,7,8) #tuple
var_4 = {'d','e','f'} #set
%store var_1
%store var_2
%store var_3
%store var_4

 Stored 'var_1' (list)
 Stored 'var_2' (dict)
 Stored 'var_3' (tuple)
 Stored 'var_4' (set)

Then on a different IPython notebook it will be sufficient to type:

%store -r var_1 
%store -r var_2 
%store -r var_3 
%store -r var_4
gibbone
  • 2,300
  • 20
  • 20
  • Any idea how you can suppress the the %store 'var1' (list) message from occuring? – BillyJo_rambler Jul 20 '18 at 15:05
  • 1
    @BillyJo_rambler you can do it by using the [`%%capture` cell magic](https://ipython.readthedocs.io/en/stable/interactive/magics.html#cellmagic-capture): just add it on top of the cell and it should suppress the message. – gibbone Jul 21 '18 at 09:07
2

If your data is in a single variable then have a try at saving it to a file using the %save magic in one notebook and then reading it back in another.

The one difficulty is that the text file will contain the data but no variable definition so I usually contatenate it with a variable definition and then exec the result.

Tony Williams
  • 613
  • 3
  • 18
  • Definitely a possible solution. Would there be any way to do it without modifying the notebook with the variable I want? So essentially making the variable scope of one notebook the same as another -- giving notebook 1 access to notebook 2's variables, without modifying notebook 2 at all? – Kyle Siegel Jul 28 '15 at 16:48
  • This does not "share data" but writes previous IPython inputs to a text files. The `%store` solution from blue-sky does save and reads back data between notebooks (and sessions, for that matter). – Eric O. Lebigot Aug 14 '17 at 20:47
0

I believe that theoretically you should be able to do so with messaging, though I would have to dig a lot deeper to figure it out.

Why would you need this capability though?

Cristian Ciupitu
  • 20,270
  • 7
  • 50
  • 76
dmargol1
  • 397
  • 1
  • 6
  • Messaging looks like it could potentially work. Essentially, I am standing up a server for a group of people to do some data analysis with a package that I am writing. Each notebook is doing a specific analysis and can be pretty long and involved. Occasionally, some data will need to be shared between notebooks, and there no way to really predict what data will need to be shared. Does that make sense? – Kyle Siegel Jul 26 '15 at 19:15