I would like to read in several *.csv files from disk into one big dataframe including a new column of paths as a string in an efficient and clean way. Is there a way other then a for-loop?
Data are stored in the same form for different realizations. The setup is the same except the realizations have varying parameter values (and thus results) but are always store via pd.to_csv() with the same columns.
Current solution via for loop:
dfs = []
for path in paths:
df = pd.read_csv(path)
df['PATH'] = path
dfs.append(df)
concated_dfs = pd.concat(dfs)
List comprehension but missing column of file paths
concated_dfs = pd.concat([pd.read_csv(path) for path in paths])
Desired result: A concated dataframe including a columns describing the path or realization.
