1

i'm generating key-value pairs like this:

KEY                       VALUE
----                      ------
user.varname.2014-01-01   10
user.varname.2014-01-02   10
user.varname.2014-01-03   10

i'll need operations:

  • query by username
  • query by varname
  • query by date

currently i'm using mysql and is working fine splitting string to query.

  1. what about query performance? is this a good way to store and query billions of rows?
  2. maybe should I use a nosql or another database?
user3175226
  • 3,579
  • 7
  • 28
  • 47

1 Answers1

4

a (very) partial answer :

a pure key-value store like redis will have very poor performance for operations like searching by date, as there will be no indexing on this particular field (well it is not a field as you specified it, but you'd like to).

So redis would have to read ALL the keys to search for a particular date, which is obviously not efficient.

With your 3 search operation specifications, and if you want to search billions of records, I don't see how you could avoid splitting into 3 fields and index them.

Pixou
  • 1,719
  • 13
  • 23
  • this is true. reading and splitting all items to search is a poor idea. thanks – user3175226 Jan 25 '14 at 11:32
  • What are you actually looking for that you don't have in sql ? speed ? scalability ? As far as speed is concerned, you may consider http://stackoverflow.com/questions/10692398/how-do-i-make-a-mysql-database-run-completely-in-memory . If you want to scale to larger dataset, look at big data databases such as hbase – Pixou Jan 25 '14 at 12:36