1

I've been trying to make replay system. So basically when player moves, system saves his datas(movements, location, animation etc.) into JSON file. In the end of the record, JSON file may be over 50 MB. I'd want to save this data into Redis with expire date (24-48 hours).

My questions are;

  1. Is it bad to save over 50 MB into Redis with expire date?
  2. How many datas that over 50 MB can Redis handle without performance loss?
  3. If players make 500 records in 48 hours, may it be bad for Redis?
  4. How many milliseconds does it takes 50 MB data from Redis with average VDS/VPS?
Obyvante
  • 95
  • 2
  • 11
  • Does the write operations occur very frequent? – Anthony Kong Apr 26 '20 at 10:05
  • Are you keeping the JSON as string or using RedisJSON? – Guy Korland Apr 26 '20 at 10:14
  • @GuyKorland It's changeable. I'd want to use which is good but now I'm using JSON as string. – Obyvante Apr 26 '20 at 11:12
  • @AnthonyKong No, we can say it saves/gets every 15 minutes – Obyvante Apr 26 '20 at 11:12
  • which data type do you want to use ? how frequent you will you read from that key? Will you need the whole value while you are getting it ? @Obyvante – Ersoy Apr 26 '20 at 11:37
  • @Ersoy I use JSON for saving data but I can parse as string. We can say it saves a data(50MB) every 5 minute. I've to get whole value. – Obyvante Apr 26 '20 at 11:53
  • I mean which redis data type you are planning to use(it is string afais but it saving a 50 mb in a single string value is not efficient) and do you need to get the whole 50MB back or is it okay to get the data by partitions. How do you want to display it? Can you provide more details about the example sub jsons such as movements, location, animation etc. – Ersoy Apr 26 '20 at 11:59
  • @Ersoy Basically, I'd want to save JSON data into Redis but I dont't know which method for saving is better. Because of that I'm using string for now. If you consider it, I can save as a map like("player1.record", HashMap). Is it good method to save it? – Obyvante Apr 26 '20 at 12:56

1 Answers1

2

Storing a large object(in terms of size) is not a good practice. You may read it from here. One of the problem is network. You need to send 50MB payload to a redis server in a single call. Also if you save them as one big object, then while retrieving, updating it (a single field, element etc), you need to get 50 MB back from server and parse it to get a single field, update it back end send back to server. That's a serious problem in terms of network.

Instead of redis strings, you may prefer sorted sets or lists depending on your use case. If you are going to store them with timestamps and get the range of events between these timestamps, then sorted sets may be an ideal solution for you. It's good for pagination etc. One of the crucial drawback is the complexity of adding a new element is O(log(N)).

lists may also provide a good playground for your case. You may use LPUSH/RPUSH to add new events to your list, and since Redis lists are implemented with linked lists, both adding a message to the beginning or end of the list is same, O(1), which is great.

Whenever an event happens, you either call ZADD or RPUSH/LPUSH to send the events to redis. If you need to query those to you may use available functions such as ZRANGEBYSCORE or LRANGE depending on your choice.

While designing your keys you may use an identifier such as user-id just like you mentioned in the comments. You will not have the problems with lists/sorted sets like you will have in strings. But choosing which one is most suitable for your depends on your use case for reads/writes or business rules.

Here some useful links to read;

Redis data types intro

Redis data types

Redis labs documentation about data types

Ersoy
  • 8,816
  • 6
  • 34
  • 48
  • I'd like to save as JSON into Redis but as I know that's not possible. So I'm parsing JSON to string and save it. I basically use default "set" parameter. – Obyvante Apr 26 '20 at 14:37
  • @Obyvante i listed possible problems with 50mb json for both saving and getting back, still it is up to you to go with your original way. – Ersoy Apr 26 '20 at 14:43