Python problem: reading multiple json files from a folder load only one json

Question

My json file example is with this statistical markers:

mean:0,23454
min:-2,83456
max:-3,45678

…

I have a folder with 20 json files:

This is my code for reading multiple json files from a folder in Python as the question Python: Read several json files from a folder

This code loads only a single json file not multiple files and I don't know why. The problem is in this function in my opinion:

for index, js in enumerate(json_files):
with open(os.path.join(path_to_json, js)) as json_file:
json_text = json.load(json_file)

All my code:

import os, json
import pandas as pd
# this finds our json files
path_to_json = 'path/'
json_files = [pos_json for pos_json in os.listdir(path_to_json) if   pos_json.endswith('.json')]


# here I define my pandas Dataframe with the columns I want to get from the json
jsons_data = pd.DataFrame(columns=['mean','min','max','peak2peak','std','variance','kurtosis','skewness','rmsOriginalSignal','rmsFiltSignal_01','rmsFiltSignal_02','rmsFiltSignal_03'])

# we need both the json and an index number so use enumerate()

for index, js in enumerate(json_files):
    with open(os.path.join(path_to_json, js)) as json_file:
        json_text = json.load(json_file)

mean = json_text['mean']
min = json_text['min']
max = json_text['max']
peak2peak = json_text['peak2peak']
std = json_text['std']
variance = json_text['std']
kurtosis = json_text['kurtosis']
skewness = json_text['skewness']
rmsOriginalSignal = json_text['rmsOriginalSignal']
rmsFiltSignal_01 = json_text['rmsFiltSignal_01'] 
rmsFiltSignal_02 = json_text['rmsFiltSignal_02']
rmsFiltSignal_03 = json_text['rmsFiltSignal_03']
jsons_data.loc[index]=       [mean,min,max,peak2peak,std,variance,kurtosis,skewness,rmsOriginalSignal, rmsFiltSignal_01, rmsFiltSignal_02, rmsFiltSignal_03]
print(jsons_data)

This code don't give any error but loads only one json file.

score 0 · Answer 1 · answered Sep 17 '18 at 16:24

It looks to me like the json_text is overwritten each iteration.

Maybe create e.g. a list and append to that list in the loop?

For example:

json_text_list = []
for index, js in enumerate(json_files):
    with open(os.path.join(path_to_json, js)) as json_file:
        json_text_list.append(json.load(json_file))

Either that or calculate your statistics in the loop (and store the results in a list or similar).

score 0 · Answer 2 · answered Sep 17 '18 at 16:26

You're overwriting the json_text variable so you read every file, but only hold the values in the last one. This seems to be because your code is not properly indented.

You will want to indent the last lines to match the indentation of the loop like so:

for index, js in enumerate(json_files):
    with open(os.path.join(path_to_json, js)) as json_file:
        json_text = json.load(json_file)

        mean = json_text['mean']
        min = json_text['min']
        max = json_text['max']
        peak2peak = json_text['peak2peak']
        std = json_text['std']
        variance = json_text['std']
        kurtosis = json_text['kurtosis']
        skewness = json_text['skewness']
        rmsOriginalSignal = json_text['rmsOriginalSignal']
        rmsFiltSignal_01 = json_text['rmsFiltSignal_01'] 
        rmsFiltSignal_02 = json_text['rmsFiltSignal_02']
        rmsFiltSignal_03 = json_text['rmsFiltSignal_03']
        jsons_data.loc[index]=       [mean,min,max,peak2peak,std,variance,kurtosis,skewness,rmsOriginalSignal, rmsFiltSignal_01, rmsFiltSignal_02, rmsFiltSignal_03]

print(jsons_data)

A simpler example

To give an easier to visualize example:

for n in range(10):
    n_squared = n * n

print(n_squared)

Will only print 81 as the print statement is not in the loop. Like the suggested fix indenting this print will print every squared value.

for n in range(10):
    n_squared = n * n
    print(n_squared)

Output:

Python problem: reading multiple json files from a folder load only one json

2 Answers2

A simpler example