Dynamically setting dataframe name while reading csv files from a folder in python

Question

I have looked up for the answer but wasn't satisfied. I don't want to store all the csv files in a dictionary or append in a single dataframe.

I have a folder consisting of 10 csv files. I want a separate dataframe for each csv file instead.

I was able to create a dictionary consisting of key representing file name and value representing its data but I want to break that further into individual dataframes.

import pandas as pd
import numpy as np
import os 
import glob

path = r'C:\Users\.....\files'
csv_files = glob.glob(os.path.join(path_new, '*.csv'))
files_name = os.listdir(path_new)

d = {os.path.splitext(os.path.basename(f))[0] : pd.read_csv(f) for f in glob.glob(os.path.join(path, '*.csv'))}

This creates a dictionary and I can access separate dataframes by calling the key of dictionary.

I tried 2nd approach by defining a function to read csv:

def read_files(name, path_final):
    name = pd.read_csv(path_final)
    return name

for csvs in files_name:
    name = csvs.split('.')[0]
    path_final = os.path.join(path, csvs)
    read_files(name, path_final)

I tried 3rd approach but it doesn't help either:

for each_file in files_name:
    with open(os.path.join(path_new, each_file)) as f:
        each_file = pd.read_csv(f)
        print(each_file)

Is there a way to create separate dataframe dynamically?

I have referred to below links so far:

using separated names is wrong idea. First it makes problem to create variables to assign dataframe, and later it makes problem to access these variables to get dataframe. And you can't use `for`-loop to run task on all dataframes. So separated variables makes only problems. And everyone will say: don't use separated variables. And separated variables are created as values in special dictionary. `globals()['name'] = value` ;) So you still will have to work with dictionary to create and access dataframes :) — furas, Mar 22 '22 at 04:16

score 0 · Accepted Answer · answered Mar 22 '22 at 05:41

globals() returns a dictionary of current module's variables. Instead of creating a dictionary in your first approach, you can create a new variable dynamically by creating a key on globals():

for f in glob.glob(os.path.join(path, '*.csv'):
    variable_name = os.path.splitext(os.path.basename(f))[0]
    globals()[variable_name] = pd.read_csv(f)

Dynamically setting dataframe name while reading csv files from a folder in python

1 Answers1