1

Not sure if it's the best title for the question... maybe someone could rename it for me?

My question is regarding performance of reading and combining data in c# ServiceStack wrapper for Redis and how the calls work internally.

I will explain two scenarios that will hopefully yield in a final result. One scenario has the list of category id's attached to the Transaction so that the Category can be stored independently.

Question: My end goal is to retrieve all transactions that have category 'food'.

I have tried to number other points where clarity would help my understanding. Consider there being 10,000 transactions and each transaction had on average 3 categories.

Note: There is a related question at ServiceStack.Net Redis: Storing Related Objects vs. Related Object Ids however doesn't explain the efficiency.

Example A

public class Transaction
{
    public List<string> CategoryIds;
}

Example B

public class Transaction
{
    public List<string> CategoryNames;
}

Code

var transactionClient = redisClient.GetTypedClient<Transaction>();

//1. is this inefficient returning all transactions?
//   is there any filtering available at this part?
var allTransactions = transactionClient.GetAll();

//2. In the case of Example A where the categories are stored as id's
//   how would I map the categories to a transaction?
//   maybe I have a List that has a container with the Transaction associated with a
//   list of Categories, however this seems inefficient as I would have to loop 
//   through all transactions make a call to get their Categories and then 
//   populate the container datatype.

//3. If we are taking Example B how can I efficiently just retrieve the transactions
//   where they have a category of food.
Community
  • 1
  • 1
David
  • 15,150
  • 15
  • 61
  • 83

1 Answers1

0

The efficiency is less network calls vs more data. Data in Redis just gets blobbed, most of the time a single API call maps 1:1 with a redis server operation. Which means you can think about the perf implications as simply downloading a json dataset blob from a remote server's memory and deserializing it on the client - which is effectively all that happens.

In some APIs such as GetAll() it requires 2 calls, 1 to fetch all the ids in the Entity set, and the other to fetch all the records with those ids. The source code of the Redis Client is quite approachable so I recommend having a look to see exactly what's happening.

Because you've only got 3 categories, it's not that much extra data you're saving by trying to filter on the server.

So your options are basically:

  • Download the entire entity dataset and filter on the client
  • Maintain a custom index mapping from Category > Ids
  • More Advanced: Use a server-side LUA operation to apply server side filtering (requires Redis 2.6)
mythz
  • 141,670
  • 29
  • 246
  • 390
  • Thanks, none of the filtering will be handled client side. It's all wrapped up a single WCF call e.g. GetTransactions(string categoryName) and the filtering and logic for returning a specific datatype is handled by the server. I just thought looping through all transactions and populating with category data sounded like it would bloat! – David Jul 20 '12 at 18:01
  • You're using WCF and you care about efficiency? You'll likely not notice any Redis optimizations :) – mythz Jul 20 '12 at 18:16
  • I've had a look at the source code and it uses the TypeSerializer to get all items back. Where would I be looking to filter the results returned initially with the LUA operation? e.g. a transaction has an accountid associated with it, i want to only retrieve those transaction with accountid 1 that is returned from the redis function (not filtered on the server-side). I've found the LUA command in AlchemyNativeClient but not sure how to use it! – David Jul 23 '12 at 10:49
  • It should be using the JSON Serializer now, TypeSerializer was replaced long ago. Alchemy client is for a non-standard Alchemy redis-server, so is likely not what you have. LUA APIs are here https://github.com/ServiceStack/ServiceStack/blob/master/src/ServiceStack.Interfaces/Redis/IRedisClient.cs#L259 - Some info about using it is http://redis.io/commands/eval - you can use the built-in json serializer to parse the payload. – mythz Jul 23 '12 at 15:32