Json loosing data after a period of time? or is it an error in my code?

Question

Below is my code, this system is a points system, where when a user talks they get given points! This system works, however after a while when I use the ;points command it says the user is new as the try except function is being fired, as a result of a key error purposefully masked by the try except. Why does it suddenly pull a key error if it worked earlier? I have also included my JSON to help!

Discord.py main script - includes both the point giver called on message, and includes point reviewer called on ;points

  @Cog.listener()
  async def on_message(self, message):
    servername = message.guild.name
    guildid = message.guild.id
    serverid = guildid
    name = message.author
    userid = message.author.id

    points = {}
    with open("./points.json","r") as f:
      points = json.load(f)
    try:
      existingpoints = points[str(userid)][str(guildid)]
      newpoints = existingpoints + 1
      points[str(userid)][str(guildid)] = newpoints
    except:
      #points[str(userid)] = {}
      points[str(userid)][str(guildid)] = 1


    try:
      existingpoints = points[str(userid)][str(guildid)]
      if existingpoints % 50 == 0:
        await message.channel.send(f"{message.author.mention} has levelled up with {existingpoints} points!")
    except:
      pass

    with open("./points.json","w") as f:
      json.dump(points,f)

    
   



  @command(name="points", aliases=["earnings"])
  async def points(self, ctx, leaderboard=None):
    servername = ctx.guild.name
    guildid = ctx.guild.id
    serverid = guildid
    name = ctx.author
    userid = ctx.author.id
    
    points = {}
    with open("./points.json","r") as f:
      points = json.load(f)
    if leaderboard == None:
      leaderboard = ("none")
      try:
        existingpoints = points[str(userid)][str(guildid)]
        await ctx.send(f"{ctx.author.display_name} has {existingpoints} points.")
      except:
        await ctx.send(f"You are new! Never spoken a word here before! Wow :slight_smile: Begin talking to earn points, then try again!")
    else:
      embed = discord.Embed(title=f"Points Leaderboard", description= "", color=0x00ff00)
      for i in points.keys():
        points = points[i][str(guildid)]
        user = await self.get_user_info(i)
        name = user.name
        embed.add_field(name=f"{name}", value=f"{points}")
      await ctx.send(embed=embed)

JSON:

{"719588575733350551": {"777618366894571530": 12, "727236037742690354": 3, "762046745248268298": 8, "799671710965694475": 7}, "738357845543616614": {"777618366894571530": 8, "727236037742690354": 2, "762046745248268298": 3, "799671710965694475": 1, "715310006710435900": 2}, "695439575329275945": {"762046745248268298": 8}, "758272354173845536": {"762046745248268298": 5}, "438762249809821699": {"715310006710435900": 81}, "155149108183695360": {"715310006710435900": 1}, "757625487353839749": {"715310006710435900": 3}, "719770082728738867": {"715310006710435900": 3}, "789522260205240350": {"762046745248268298": 4}, "729667553491812403": {"762046745248268298": 50, "715310006710435900": 2}, "398601531525562369": {"793109512638824458": 2}, "508968886998269962": {"715310006710435900": 2}, "318567688655863810": {"394711355543781378": 1}, "408754269127180288": {"715310006710435900": 2}, "760720870934708244": {"762046745248268298": 3}, "690965766853361727": {"715310006710435900": 2}, "437808476106784770": {"799671710965694475": 1}, "648567468549472257": {"762046745248268298": 1, "799671710965694475": 2}, "705016654341472327": {"394711355543781378": 1}}

Sorry but I dont want to tell people about that, unless it is related to this issue?? — Romulus Hill, Jan 20 '21 at 12:46
It is related, some hosts (like heroku) will remove any changes made in the local file system when the app is restarted — Łukasz Kwieciński, Jan 20 '21 at 12:47
I'm not really familiar with repl.it, you should research that, try hosting the bot for a couple of hours in your machine, if the error doesn't happen the issue is with the host — Łukasz Kwieciński, Jan 20 '21 at 13:08

score 0 · Accepted Answer · answered Jan 22 '21 at 14:29

You're losing the data inside because you're treating the files poorly. Try closing the JSON file after opening it. Not closing files is bad practice, it can slow don't your program and the data is often cached in memory and doesn't hit the hard drive until the file is closed. The longer you keep the file open, the greater the chance that you will lose data. You also open the JSON files in on_message which is incredibly dangerous. If you do that you could hit the limit to how many files you can have open at once.

Json loosing data after a period of time? or is it an error in my code?

1 Answers1