1

I am trying to create a series of dictionaries from CSVs that I want to import but I am not sure the best way to do it.

I used RatingFactors = os.listdir(RatingDirectory) and

CSVLocations = [] for factor in RatingFactors: CSVLocations.append(RatingDirectory + factor)

to create a list of CSVs, these CSVs contain what is essentially a dictionary of FactorName | Factor Value, then 1 | 5, 2 | 3.5.

I want to create a dictionary for each CSV, ideally named based on the CSVs name. However I understand that when looping across variables it is considered bad to try and name my variables inside the loop.

I tried creating a generator function using df_from_each_file = (pd.read_csv(CSVs) for CSVs in CSVLocations)

and if I print the generator using for y in df_from_each_file: print(y) it gives me each of the dataframes but I don't know how to separate them out?

What is the Pythonic way to do this?

How the CSVs look post import

0         0  1.1
1         1  0.9
2         2  0.9
3         3  0.9
etc

Edit:

Attempt to rephrase my question.

I have a series of CSVs which look like they are formatted like dictionaries, they have two columns and they represent how one factor relates to another. I would like to make a dictionary for each CSV, named like the CSV so that I can interact with them from Python.

Edit 2:

I believe this question is different than the one referenced as that is creating a single dataframe which contains all of the dictionaries, I want all of the dictionaries to be separate rather than in a single unit. I tried using their answer before asking this and I could not separate them out.

Violatic
  • 374
  • 2
  • 18
  • 3
    Format your question properly please. – Sheldore Sep 02 '18 at 15:19
  • 2
    What @Bazingaa said, and also please include an example of how the CSVs are structured in your input – Kwright02 Sep 02 '18 at 15:20
  • I'm not sure how I can better explain how they're formatted. I have tried to include an example – Violatic Sep 02 '18 at 15:42
  • Possible duplicate of [Import multiple csv files into pandas and concatenate into one DataFrame](https://stackoverflow.com/questions/20906474/import-multiple-csv-files-into-pandas-and-concatenate-into-one-dataframe) – Andrey Portnoy Sep 02 '18 at 16:26

1 Answers1

2

I think need dict comprehension with basename for keys:

import glob, os

files = glob.glob('files/*.csv')
sers={os.path.basename(f).split('.')[0]:pd.read_csv(f,index_col=[0]).squeeze() for f in files}

If want one big Series:

d = pd.concat(sers, ignore_index=False)
jezrael
  • 822,522
  • 95
  • 1,334
  • 1,252
  • So this seems to almost work and so I'm probably misunderstanding. This gives me one dictionary called DFS which contains sub dictionaries? If I'm looking to separate these out how would I do that? – Violatic Sep 02 '18 at 16:13
  • @Violatic - You can use `dfs['columns_name']` for DataFrame – jezrael Sep 02 '18 at 16:14
  • My only concern is that what you're asking means that I need to reference dfs when needed rather than the dictionary I want? Perhaps what I'm asking isn't actually helpful. – Violatic Sep 02 '18 at 16:27
  • @Violatic - Yes, exactly. If want reference each `Series` use `sers['file_name']`. I add new parameter `index_col` with `squeeze` for return `Series` and if want one big `Series` use `concat` – jezrael Sep 02 '18 at 16:33