Append New Data to JSON

Question

I'm struggling to append new data to a JSON. I simply want to call JSON data from an API, and put it into a JSON file under "matches". I am creating a blank JSON file containing the following and just going from there.

{
    "matches": [
    ]
}

Here is my code, though I suspect only the last line is important:

print ("No records found...\n Creating new match history bank.")
file_handle = open(all_matches_index, "w+")
file_handle.write('{\n    "matches": [\n    ]\n}')

for game_id in game_ids:
        full_match_data = watcher.match.by_id(my_region, game_id)
        #this is the problem line:
        file_handle.write(json.dumps({"matches" : full_match_data }, sort_keys=True, indent = 4, separators=(',', ': ')))

I have tried a litany of different solutions and nowhere online seems to address this, or at least I am not understanding what I am reading. I know its a simple problem but I can't solve it.

Some examples of what I have tried:

file_handle.write(json.dumps(full_match_data["matches"], sort_keys=True, indent = 4, separators=(',', ': ')))

file_handle["matches"].write(json.dumps(full_match_data, sort_keys=True, indent = 4, separators=(',', ': ')))

file_handle["matches"] = {**full_match_data, file_handle["matches"] sort_keys=True, indent = 4, separators=(',', ': ')))}

file_handle.write(json.dumps({"matches" : [full_match_data]}, sort_keys=True, indent = 4, separators=(',', ': ')))

file_handle.write(json.dumps(["matches" {full_match_data}], sort_keys=True, indent = 4, separators=(',', ': ')))

Edit 1: Changes based on the response from Pranav Hosangadi,

First of all, thank you for your response, it contains far more information than a simple fix and this really helps me as I learn.

I changed my code so that it now looks as follows:

file_handle = open(all_matches_index, "w+")
file_handle.write('{\n    "matches": [\n    ]\n}')
file_handle.close()


matches = []
for game_id in game_ids:
    full_match_data = watcher.match.by_id(my_region, game_id)
    matches.append(full_match_data)

with open(all_matches_index, "w") as file_handle:
    json.dump(file_handle, {"matches": matches})
    
    file_handle.close

Unfortunately, it doesn' work. It seems to think for quite a while (being an intel pentium) and then returns an empty file?

Running it a second time fills the file with:

{
    "matches": [
    ]
}

Would you be able to tell me where I am going wrong?

Once I get it to work I will switch to the pythonic way of doing it. Many thanks.

Pranav Hosangadi · Accepted Answer · 2020-09-05T17:42:56.703

Why write to the file every time you get a result from the API? Just append new full_match_data to a list and write it once after you have them all. Also note that json.dump can write to a file handle directly, no need to dumps into a string and then write that string to a file.

matches = []
for game_id in game_ids:
    full_match_data = watcher.match.by_id(my_region, game_id)
    matches.append(full_match_data)

with open(all_matches_index, "w") as file_handle:
    json.dump({"matches": matches}, file_handle)

You can replace the loop with a list comprehension for more pythonic way:

matches = [watcher.match.by_id(my_region, game_id) for game_id in game_ids]

Re. edits in question:

There was an error in my original answer. json.dump expects json.dump(data, file_handle). See the updated answer.

file_handle = open(all_matches_index, "w+")
file_handle.write('{\n    "matches": [\n    ]\n}')
file_handle.close()

This part of your code is unnecessary. All it does is write an empty file. The with... block in the code later is supposed to overwrite this anyway.

matches = []
for game_id in game_ids:
    full_match_data = watcher.match.by_id(my_region, game_id)
    matches.append(full_match_data)

This is the part that creates the list

with open(all_matches_index, "w") as file_handle:
    json.dump({"matches": matches}, file_handle)

This is the part that writes the data to the file. Using with creates a context manager that automatically closes the file for you once the with block ends. You can find lots of information on the web.

    file_handle.close

This is (a) unnecessary because the context manager handled closing the file anyway, and (b) wrong, because you need to call the function, so file_handle.close() like you did earlier.

@PranavHosangadi I tried to implement your solution but have come unstuck. Would you be able to review my edited question? — Solebay Sharp, Sep 04 '20 at 22:51
@PranavHosangadi thank you, your help is deeply appreciated. — Solebay Sharp, Sep 07 '20 at 17:55

score 1 · Answer 2 · answered Sep 04 '20 at 20:48

When you load a JSON object in python it is typically just a dictionary. You can add data to a JSON the same way you would add data to a dictionary. It seems to me like you want all of the matches to be under the key matches in your final file. If that is the case, you can create a list or dictionary of all of the matches, then add them under the matches key, like this

match_dict= {'match_1':match_1,'match_2':match_2,...,'match_n':match_n}
matches={'matches':match_dict}

If you want to do this with an existing JSON file, just read that file in with JSON package, and then do the above, and save the file at the end.

Append New Data to JSON

2 Answers2