How to merge multiple json files into one file in python

Question

I want to merge multiple json files into one file in python. The thing that I want to do is if there are several .json files like:

# temp1.json
[{'num':'1', 'item':'smartphone','data':'2019-01-01'},
{'num':'2', 'item':'smartphone','data':'2019-01-02'},
{'num':'3', 'item':'smartphone','data':'2019-01-03'},
{'num':'4', 'item':'smartphone','data':'2019-01-04'}]

# temp2.json
[{'num':'5', 'item':'smartphone','data':'2019-01-05'},
{'num':'6', 'item':'smartphone','data':'2019-01-06'},
{'num':'7', 'item':'smartphone','data':'2019-01-07'}]

# temp3.json
[{'num':'8', 'item':'smartphone','data':'2019-01-08'},
{'num':'9', 'item':'smartphone','data':'2019-01-09'},
{'num':'10', 'item':'smartphone','data':'2019-01-10'},
{'num':'11', 'item':'smartphone','data':'2019-01-11'},
{'num':'12', 'item':'smartphone','data':'2019-01-12'}]

The result.json files I want to get should look like:

# result.json
[{'num':'1', 'item':'smartphone','data':'2019-01-01'},
{'num':'2', 'item':'smartphone','data':'2019-01-02'},
{'num':'3', 'item':'smartphone','data':'2019-01-03'},
{'num':'4', 'item':'smartphone','data':'2019-01-04'},
{'num':'5', 'item':'smartphone','data':'2019-01-05'},
{'num':'6', 'item':'smartphone','data':'2019-01-06'},
{'num':'7', 'item':'smartphone','data':'2019-01-07'},
{'num':'8', 'item':'smartphone','data':'2019-01-08'},
{'num':'9', 'item':'smartphone','data':'2019-01-09'},
{'num':'10', 'item':'smartphone','data':'2019-01-10'},
{'num':'11', 'item':'smartphone','data':'2019-01-11'},
{'num':'12', 'item':'smartphone','data':'2019-01-12'}]

The result.json files I got is:

# result.json
[[{'num':'1', 'item':'smartphone','data':'2019-01-01'},
{'num':'2', 'item':'smartphone','data':'2019-01-02'},
{'num':'3', 'item':'smartphone','data':'2019-01-03'},
{'num':'4', 'item':'smartphone','data':'2019-01-04'}],
[{'num':'5', 'item':'smartphone','data':'2019-01-05'},
{'num':'6', 'item':'smartphone','data':'2019-01-06'},
{'num':'7', 'item':'smartphone','data':'2019-01-07'}],
[{'num':'8', 'item':'smartphone','data':'2019-01-08'},
{'num':'9', 'item':'smartphone','data':'2019-01-09'},
{'num':'10', 'item':'smartphone','data':'2019-01-10'},
{'num':'11', 'item':'smartphone','data':'2019-01-11'},
{'num':'12', 'item':'smartphone','data':'2019-01-12'}]]

I used the code to merge .json files from here and changed it very slightly like below:

files=['my.json','files.json',...,'name.json']

def merge_JsonFiles(filename):
    result = list()
    for f1 in filename:
        with open(f1, 'r') as infile:
            result.append(json.load(infile))

    with open('counseling3.json', 'w') as output_file:
        json.dump(result, output_file)

merge_JsonFiles(files)

I already read several related questions, but there is no answer I need. Can anyone help me?

use extend instead of append. – Paul Rooney Aug 09 '19 at 02:18 — Paul Rooney, Aug 09 '19 at 02:18
function names should be lowercase – Michael Dec 01 '22 at 13:47 — Michael, Dec 01 '22 at 13:47

score 18 · Accepted Answer · answered Aug 09 '19 at 02:18

You should use extend instead of append. It will add the items of the passed list to result instead of a new list:

files=['my.json','files.json',...,'name.json']

def merge_JsonFiles(filename):
    result = list()
    for f1 in filename:
        with open(f1, 'r') as infile:
            result.extend(json.load(infile))

    with open('counseling3.json', 'w') as output_file:
        json.dump(result, output_file)

merge_JsonFiles(files)

score 6 · Answer 2 · edited Aug 06 '20 at 09:25

6

import json
import pandas as pd

with open('example1.json') as f1:               # open the file
    data1 = json.load(f1)

with open('example2.json') as f2:                # open the file       
    data2 = json.load(f2)
    
df1 = pd.DataFrame([data1])                      # Creating DataFrames
df2 = pd.DataFrame([data2])                      # Creating DataFrames

MergeJson = pd.concat([df1, df2], axis=1)         # Concat DataFrames

MergeJson.to_json("MergeJsonDemo.json")          # Writing Json

edited Aug 06 '20 at 09:25

Gryu

2,102
2
16
29

answered Aug 06 '20 at 08:34

Anjali Bhagat

61
1
1

2

I love how clean this looks from a code readability standpoint. – Ryan Hart Jun 06 '21 at 02:02
1

Arguments: ('ValueError', ValueError("DataFrame columns must be unique for orient='columns'.")) – Dmitrij Holkin Jan 27 '22 at 07:21

score 5 · Answer 3 · answered Apr 23 '22 at 09:11

5

files=['my.json','files.json',...,'name.json']

with open('merged_file_name.json', "w") as outfile:
   outfile.write('{}'.format('\n'.join([open(f, "r").read() for f in files])))

answered Apr 23 '22 at 09:11

Abhi

51
1
2

4

When answering a question it is better to write some description in addition to code. – keiv.fly Apr 23 '22 at 14:29

YScharf · Answer 4 · 2022-06-02T20:21:11.407

0

If the goal is to merge several JSON files, try the following:

def combine_jsons():
    file_list = ['first.json', 'second.json',... ,'last.json']
    all_data_dict = {}
    for json_file in file_list:
       with open(json_file,'r+') as file:
           # First we load existing data into a dict.
           file_data = json.load(file)
       all_data_dict.update(file_data)
    with open('merged_data.json', "w") as outfile:# save to json file
        json.dump(all_data_dict, outfile)

If you it must be a list for some reason just change the last line

json.dump([all_data_dict], outfile)

Based on How do I merge two dictionaries in a single expression (take union of dictionaries)?

edited Jun 02 '22 at 20:21

answered Feb 28 '22 at 15:52

YScharf

1,638
15
20

this givers error – lord stock Jun 02 '22 at 12:48
What error? Indentation? fixed! – YScharf Jun 02 '22 at 20:20

Talex Maxim · Answer 5 · 2023-04-19T14:54:42.060

0

def merge_files(self):
    res = list()
    for f1 in self.listDomainToMerge:
        item = json.load(open(f1, encoding="utf-8"))
        res.append(item)

    with open('../Configuration%s/configuration.json' % self.owner, 'w') as output:
        output.write(json.dumps(res, ensure_ascii=False, indent=2))

edited Apr 19 '23 at 14:54

answered Apr 19 '23 at 14:53

Talex Maxim

1
1

score -1 · Answer 6 · answered Aug 09 '19 at 03:10

There is another way.Just loads the json text in those files as a python list, and add them together. Code as below.

# temp1.json
json_a = [{'num':'1', 'item':'smartphone','data':'2019-01-01'},
{'num':'2', 'item':'smartphone','data':'2019-01-02'},
{'num':'3', 'item':'smartphone','data':'2019-01-03'},
{'num':'4', 'item':'smartphone','data':'2019-01-04'}]

# temp2.json
json_b = [{'num':'5', 'item':'smartphone','data':'2019-01-05'},
{'num':'6', 'item':'smartphone','data':'2019-01-06'},
{'num':'7', 'item':'smartphone','data':'2019-01-07'}]

# temp3.json
json_c = [{'num':'8', 'item':'smartphone','data':'2019-01-08'},
{'num':'9', 'item':'smartphone','data':'2019-01-09'},
{'num':'10', 'item':'smartphone','data':'2019-01-10'},
{'num':'11', 'item':'smartphone','data':'2019-01-11'},
{'num':'12', 'item':'smartphone','data':'2019-01-12'}]

print(json_a + json_b + json_c)

Output:

[{'num': '1', 'item': 'smartphone', 'data': '2019-01-01'},
 {'num': '2', 'item': 'smartphone', 'data': '2019-01-02'},
 {'num': '3', 'item': 'smartphone', 'data': '2019-01-03'},
 {'num': '4', 'item': 'smartphone', 'data': '2019-01-04'},
 {'num': '5', 'item': 'smartphone', 'data': '2019-01-05'},
 {'num': '6', 'item': 'smartphone', 'data': '2019-01-06'},
 {'num': '7', 'item': 'smartphone', 'data': '2019-01-07'},
 {'num': '8', 'item': 'smartphone', 'data': '2019-01-08'},
 {'num': '9', 'item': 'smartphone', 'data': '2019-01-09'},
 {'num': '10', 'item': 'smartphone', 'data': '2019-01-10'},
 {'num': '11', 'item': 'smartphone', 'data': '2019-01-11'},
 {'num': '12', 'item': 'smartphone', 'data': '2019-01-12'}]

How to merge multiple json files into one file in python

6 Answers6

Linked