Jupyter Multiple Notebooks using Same Data

Question

I have built out a Jupyter notebook for my previous analysis.
And I want to start a different branch of analysis, using the some of the same dataframes from previous analysis.

How do I use the previous dataframes without coping all my code to rebuild my previous analysis, in my new notebook?

Answer 1

You can share data across notebooks with Jupyter magics. For example:

Given

# Notebook 1
import pandas as pd


d = {"one" : pd.Series([1., 2., 3.], index=list("abc"))}
df = pd.DataFrame(d)

Code

%store df

Recall the DataFrame in a separate notebook:

# Notebook 2
%store -r df
df

Output

More on this in the older IPython docs. See also Jupyter's %bookmark magic for sharing directories.

Answer 2

You can pickle the dataframe then load the dataframe in your new notebook. This is fine for short term data reuse. For long term data storage, writing then reading a text csv file may be more reliable.

pickle_save.py

import os
import pandas as pd

pickle_location = r'd:\temp\pickle_file'
df = pd.DataFrame({'A':1,'B':2}, index=[0])
df.to_pickle(pickle_location)
if os.path.exists(pickle_location):
    print('pickle created')

pickle_load.py

import os
import pandas as pd

pickle_location = r'd:\temp\pickle_file'
df_load = pd.read_pickle(pickle_location)
print(df_load)