0

I'm using ServiceStack.Redis to implement a demo project. It contains two POCOs i.e Albums and its Songs.

Below is the search results measured using a stopwatch instance:

Time elapsed searching 5804 items is 00:00:00.1243984 <-- Albums
Time elapsed searching 138731 items is 00:00:02.0592068 <-- Songs

As you can see the search for the songs is taking too much time. I'm displaying the results in a WPF application wherein the search term is also entered. The lag is a no-go for redis.

Below is the code used for searching:

IEnumerable<int> songsFromRedis =
    songRedis.GetAll()
    .Where(song => song.Title != null 
        &&  song.Title.ToLowerInvariant().Contains(searchText))
    .OrderBy(song => song.Title)
    .Select(x => x.AlbumId);

If we cannot make it any faster, would ElasticSearch help ?

Dev
  • 987
  • 2
  • 14
  • 32
  • 1
    In a word: yes. Take a look at Elasticsearch and you should also look into using the [Elasticsearch .NET client](https://github.com/elasticsearch/elasticsearch-net). Also, [Elasticsearch: The Definitive Guide](http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/) (published book) is completely free, which should help to understand concepts. – pickypg Nov 20 '14 at 00:38

2 Answers2

2

The issue is how you're using Redis, i.e. songRedis.GetAll() downloads the entire dataset, deserializes all entities into C# objects and performs the search on the client.

You should never download and query an entire dataset across the network on the client (i.e with any datastore), even a full server-side table-scan query would perform much better since only the filtered results are returned to the client and not the entire dataset. Ideally even full server-side table-scans should be avoided and any queries should be made via an index.

Redis doesn't have support for indexes built-in, but when needed you can use a SET to manually create indexes between entities in Redis.

Community
  • 1
  • 1
mythz
  • 141,670
  • 29
  • 246
  • 390
1

Any search oriented database would help. Even mysql fulltext search, which is notoriusly slow, would be a lot better here.

Elasticsearch is one good alternative, Sphinx another good one. ES has the easy scalability and ease of use, sphinx has the performance win and otherwise most of the common features but is a bit more work to learn and to scale.

Clarence
  • 2,944
  • 18
  • 16
  • Thanks. But, is the redis search times above, expected or am I doing it wrong ? – Dev Nov 20 '14 at 02:38
  • mythz answer below is basically the answer to this question. However I would not recommend you to try to create your own search index in redis. It's not the "redis search time" thats slow, it's pulling all the data out of redis before searching it that's the problem. – Clarence Nov 20 '14 at 08:19
  • So, should I execute GetAll() once at app start, store it in cache and then search it instead of Redis ? This defeats the purpose of using redis itself. Can we instead get only the items that match the search criteria ? – Dev Nov 20 '14 at 08:27
  • Getting it once and saving it in a cache is better than getting it in realtime, but if you get a large ammount of data to search through it will also soon start to perform badly. The purpose of Redis should never be to have a quick search function. The search indices in elasticsearch or sphinx are optimized to be searched, redis is definitely not and object searches in C# will not scale whatsoever. – Clarence Nov 20 '14 at 09:59