0

I want to store dataframes in a nested dictionary in the following manner which categorizes the dataframes according to the velocity (C2 and C5, respectively) and measurement spot (-45.0 and -50.0, respectively) as well as their time of creation:

data_dict = {velocity:{spot:{"time_of_creation":df}}}

e.g. {"C2":{"-45.0":{"29.06.2022 19:30:00":df}}

The file system is the following:

data/C2/spot_-45.0/
data/C2/spot_-50.0/
data/C5/spot_-45.0/
data/C5/spot_-50.0/

my code:

import pandas as pd
import os
import math
import csv
import numpy as np
from datetime import datetime


def import_data(filename):
   
   with open(filename, encoding='latin') as file: 
       data = file.read(200)
       skip = 0
       custom_columns = ["x", "y", "z"]
 
   import_parameters = {
       'skiprows' : skip,
       'header' : None,
       'names' : custom_columns,
       'delimiter' : ' ',
       'index_col' : False,
       'decimal' : '.',
       }
   
   data = pd.read_csv(filename, **import_parameters)
   
   return data


path = "data/"
velocities = ["C2", "C5"]
spots = ["-45.0", "-50.0"]
data_dict = {}

for velocity in velocities:
   for spot in spots: 
       files_list = sorted([i for i in os.listdir (path + str(velocity) + "/spot_" + str(spot)) if i.endswith(".csv")])
       times = []
       for filename in files_list:
           t_unix = os.path.getmtime(path + str(velocity) + "/spot_" + str(spot) + "/" + filename) + 7200
           t_cet = datetime.utcfromtimestamp(t_unix).strftime('%Y-%m-%d %H:%M:%S')
           times.append(t_cet)
           for time, filename in zip(times, files_list):
               data_dict [str(velocity)][str(spot)][time] = import_data(path + str(velocity) + "/spot_" + str(spot) + "/" + filename)

How do I set up data_dict before the for loop operation? Neither data_dict = {{{}}} nor data_dict = {} works. In a previous attempt, the code above created the dictionary in the desired way, but only with entries for "C2" and "-45.0". The iteration for "C5" and "-50.0" was not performed. How can this be solved?

Thank you very much for your help.

Barmar
  • 741,623
  • 53
  • 500
  • 612
ph1lipp
  • 1
  • 1
  • You need to create each level of the nesting before you can add elements to it. `collections.defaultdict()` can be helpful in doing this automatically. – Barmar Aug 05 '22 at 21:15
  • See https://stackoverflow.com/questions/19189274/nested-defaultdict-of-defaultdict for how to use `defaultdict` for multiple levels. – Barmar Aug 05 '22 at 21:16
  • Thank you, it worked using: `class NestedDefaultDict(defaultdict): def __init__(self, *args, **kwargs): super(NestedDefaultDict, self).__init__(NestedDefaultDict, *args, **kwargs) def __repr__(self): return repr(dict(self)) data_dict = NestedDefaultDict()` – ph1lipp Aug 06 '22 at 09:54

0 Answers0