0

I would like to be able to iterate over all data frames (not the names!) that are currently in the workspace.

To get a list of all data frames I found the following solution here:

import pandas as pd

# create dummy dataframes
df1 = pd.DataFrame({'Col1' : list(range(100))})
df2 = pd.DataFrame({'Col1' : list(range(100))})

# check whether all variables in scope are pandas dataframe. 
# Dir() will return a list of string representations of the variables. 
# Simply evaluate and test whether they are pandas dataframes
alldfs = [var for var in dir() if isinstance(eval(var), pd.core.frame.DataFrame)]

print(alldfs) # df1, df2

This works as described, but the problem is that I would like to be able to use this list to iterate over the actual data frames, not the names of the data frames.

The following code returns the length of the names of the data frames but should return the length of the data frames (number of lines):

for df in alldfs:
    print(len(df))

It should return:

100
100

It returns:

3
3

How can I fix this?

mrvideo
  • 39
  • 7

2 Answers2

2

This will yield the output you want as dict:

import pandas as pd

df1 = pd.DataFrame({'Col1' : list(range(100))})
df2 = pd.DataFrame({'Col1' : list(range(100))})

alldfs = {key: value for key, value in locals().items() if isinstance(value, pd.core.frame.DataFrame)}

#or using your method

alldfs = {var: eval(var) for var in dir() if isinstance(eval(var), pd.core.frame.DataFrame)}

print(alldfs) 

for df in alldfs.values():
    print(len(df))

Output:

{'df1':     Col1
0      0
1      1
2      2
3      3
4      4
..   ...
95    95
96    96
97    97
98    98
99    99

[100 rows x 1 columns], 'df2':     Col1
0      0
1      1
2      2
3      3
4      4
..   ...
95    95
96    96
97    97
98    98
99    99

[100 rows x 1 columns]}
100
100
adnanmuttaleb
  • 3,388
  • 1
  • 29
  • 46
0

You need to use an eval on the elements of your list, because those are the names of your dataframe variables. i.e :

for df in alldfs:
    print(len(eval(df)))
Cshitiz Dhingra
  • 155
  • 1
  • 11