Redis key partitioning practices with linked items

Question

I'm using a Redis database and ServiceStack client for it. I have a class called "Post" which has a property GroupId. Now when I'm storing this class the key is "urn:post:2:groupid:123". Now if I want to find all posts related to one group i need to use SearchKeys("urn:*groupid:123") method to retrieve all posts related to one group. Is this best practice to use Redis DB or should I convert my post key into form of "urn:groupid:123"post:2" ? If so how I can achieve this?

Post class:

  public class Post
    {
        public const string POST_INCREMENT_KEY =  "POST_INCREMENT";

        public string Id { get; set; }
        public string Message { get; set; }
        public string GroupId { get; set; }

        public void BuildId(long uniqueId)
        {
            Id = uniqueId  + ":groupid:" + GroupId;
        }
    }

Code for storing post:

var post = new Post
                {
                    GroupId = groupId,
                    Message = Request.Form["message"]
                };

                post.BuildId(_client.Increment(Post.POST_INCREMENT_KEY, 1));
                _client.Store(post);

score 1 · Accepted Answer · edited May 23 '17 at 12:05

The best practice in redis is to maintain an index of the relationship you want to query.

Manually maintaining an index in Redis

An index is just a redis SET containing the related Ids you want to maintain, given that you want to "retrieve all posts related to one group" I would maintain the following index:

const string GroupPostIndex = "idx:group>post:{0}";

So that everytime you store a post, you also want to update the index, e.g:

client.Store(post);
client.AddItemToSet(GroupPostIndex.Fmt(groupId), post.Id);

Note: Redis SET operations are idempotent in that adding an item/id multiple times to a SET will always result in there being only one occurrence of that item in the SET, so its always safe to add an item to the set whenever storing a POST without needing to check if it already exists.

Now when I want to retrieve all posts in a group I just need to get all the ids from the SET with:

var postIds = client.GetAllItemsFromSet(GroupPostIndex.Fmt(groupId));

Then fetch all the posts with those ids:

var posts = redis.As<Post>().GetByIds(postIds);

Using ServiceStack.Redis Related Entities API's

The above shows what's required to maintain an index in Redis yourself, but as this is a common use-case, ServiceStack.Redis also offers a high-level typed API that you can use instead.

Which lets you store related entities with:

client.As<Group>().StoreRelatedEntities(groupId, post);

Note: this also takes care of storing the Post

and retrieve them with:

var posts = client.As<Group>().GetRelatedEntities<Post>(groupId);

It also offers other convenience API's like quickly finding out how many posts there are within a given group:

var postsCount = client.As<Group>().GetRelatedEntitiesCount<Post>(groupId);

and deleting either 1 or all entities in the group:

client.As<Group>().DeleteRelatedEntity<Post>(groupId, postId);
client.As<Group>().DeleteRelatedEntities<Post>(groupId); //all group posts

Interesting... I got a nullrefence exception from StoreRelatedEntities method when i had a property called Id in my Post and its type was string. Fixed it by changing type to ulong — Panu Oksala, Dec 05 '14 at 17:27
@Mino hmmm, I haven't been able to repro this with String Ids [in this commit](https://github.com/ServiceStack/ServiceStack.Redis/commit/7bbc3fdce7557ee099212cfb9916885b763b581f). Can you add a [gist](https://gist.github.com/) that shows a repro? — mythz, Dec 05 '14 at 18:04
Actually it happen because Id was null (I didn't set any value into it). — Panu Oksala, Dec 05 '14 at 18:20
Btw. is there an easy way to fetch related entities in specified order? — Panu Oksala, Dec 05 '14 at 18:23
@Mino Not by default as redis `SET` is an unordered collection, you would need to add an order on the model and order it on the client. — mythz, Dec 05 '14 at 18:33

Redis key partitioning practices with linked items

1 Answers1

Manually maintaining an index in Redis

Using ServiceStack.Redis Related Entities API's