What are all the different ways of storing data in a .json file using python?

Question

I've been currently working on a warn system for my discord.py bot. I have been going pretty well until I realized I don't know how to add to the file instead of overwriting the file. I don't want to add to the end of the file because I want it to be categorized by discord guilds, then the guilds members. Therefore when someone wants to check their warnings, it will show only their warnings in the specific guild the command is called in. My question is what are all the different ways of storing the information to the .json file?

Ex: warn_list[str(ctx.guild.id)][member.id].add({stuff in here})

As shown in the example I don't want to know the different modes of storing aka 'a', 'w', ect. I want to know, if there are any, what can be added to the end. Like .append() or .add(). I've looked for a list on google but I can't find the info I am looking for. Any help would be much appreciated.

Sincerely, killrebeest

Answer 1

How to Add Custom Data that is Persisted to JSON

You are adding to a dictionary, so you would use dict[key] = value to add data, and value = dict[key] to read data ¹ . To add to the end of a list, use append().

You're pretty close with what you had. Let's refactor for mental clarity:

warns_for_a_user = warn_list[str(ctx.guild.id)][member.id]

We can directly use this dictionary for storing warning information about this member.id. We need a data structure and I think a list fits the bill (or a dict). You might want time information in these warnings, and time seems like a sensible sort order and/or primary key.

I want to know, if there are any, what can be added to the end

One usually does not modify a .json file directly. The only robust way to modify a .json file is to slurp the entire contents into memory, then save it back to .json after modification.

Think of a .json file as a photograph or snapshot in time. If the scene has changed, you simply throw away the old photograph and take another photograph; if state has updated, we simply junk the old warnings.json and create a new warnings.json.

Step 1: Add New Field and Append to New Field

Given that we have information on the latest warning, we can persist this to the user's history:

# working with givens:
warning = { "type": "spam", "count": "20", "timestamp": 1612756912.987963 }

if not history in warns_for_a_user:
    warns_for_a_user['history'] = []
warns_for_a_user['history'].append([warning['timestamp'], warning])

These new fields that you add should not conflict with anything because they are bespoke fields that are unknown to other parts of code.

If you find it convoluted to wrap each warning in a timestamp, that is because it is. However, it is common to choose a single field (or combination of fields) to be the representative key for the entire object. This helps humans reading the json file directly, and also can simplify the processing logic.

Step 2: Read New Field Data

Later, the user wants to view his warnings. The following prints each warning one at a time (datetime code from SO):

import datetime
for epoch_time, warn in warns_for_a_user['history']:
    _datetime = datetime.datetime.fromtimestamp(epoch_time)
    _human_datetime = _datetime.strftime('%Y-%m-%d %H:%M:%S')
    print(f"{_human_datetime}: [{warn['count']}x] {warn['type']}")

Output:

2021-02-07 20:01:52: [20x] spam

Supported Data Types

The JSON encoder docs has a mapping of how it translates serialized JSON data into deserialized python objects. Note that it converts tuple to list.

Inspecting warnings.json

If you followed my example, you should see the following added under the appropriate user in warnings.json:

{"history": [[1612756912.987963, {"count": 20, "type": "spam", "timestamp": 1612756912.987963}]]}

¹ square bracket notation is syntactic sugar for __getitem__() and __setitem__()