56

A number of sources, including the official Redis documentation, note that using the KEYS command is a bad idea in production environments due to possible blocking. If the approximate size of the dataset is known, does SCAN have any advantage over KEYS?

For example, consider a database with at most 100 keys of the form data:number:X where X is an integer. If I want to retrieve all of these, I might use the command KEYS data:number:*. Is this going to be significantly slower than using SCAN 0 MATCH data:number:* COUNT 100? Or are the two commands essentially equivalent in this circumstance? Would it be accurate to say that SCAN is preferable to KEYS because it protects against the scenario where an unexpectedly large set would be returned?

Jake
  • 7,565
  • 6
  • 55
  • 68

3 Answers3

63

You shouldn't care about current command execution but about the impact to all other commands, since Redis processes commands using a single thread (i.e. while a command is being executed all others need to await until executing one ends).

While keys or scan might provide you similar or identical performance executed alone in your case, some milliseconds blocking Redis will significantly decrease overall I/O.

This the main reason to use keys for development purposes and scan on production environments.

OP said:

"While keys or scan might provide you similar or identical performance executed alone in your case, some milliseconds blocking Redis will significantly decrease overall I/O." - This sentence seems to indicate that one command blocks Redis, and the other doesn't, which can't be the case. If I am guaranteed 100 results from my call to KEYS, in what way is it worse than SCAN? Why do you feel that one command is more prone to blocking?

There should be a good difference when you can paginate the search. It's not the same being forced to get 100 keys in a single pass than being able to implement pagination and get 100 keys, 10 by 10 (or 50 and 50). This very small interruption can let other commands sent by the application layer be processed by Redis. See what Redis official documentation says about this:

Since these commands allow for incremental iteration, returning only a small number of elements per call, they can be used in production without the downside of commands like KEYS or SMEMBERS that may block the server for a long time (even several seconds) when called against big collections of keys or elements

.

Matías Fidemraizer
  • 63,804
  • 18
  • 124
  • 206
  • 3
    Very true. However, with only 100 keys it is likely to have no practical difference. Also, one should note that `SCAN`'s `COUNT` argument is only a hint rather than a directive. – Itamar Haber Sep 16 '15 at 09:41
  • 3
    @ItamarHaber BTW I like to architect solutions with right approaches by design instead of relying on "this won't never happen". In the other hand, don't you think that if you need a `keys` or `scan` in the global keyspace it might sound like there're better solutions to store the data and access it by pages? `lrange`, `zrange` :D – Matías Fidemraizer Sep 16 '15 at 09:56
  • 1
    "While keys or scan might provide you similar or identical performance executed alone in your case, some milliseconds blocking Redis will significantly decrease overall I/O." - This sentence seems to indicate that one command blocks Redis, and the other doesn't, which can't be the case. If I am guaranteed 100 results from my call to `KEYS`, in what way is it worse than `SCAN`? Why do you feel that one command is more prone to blocking? – Jake Sep 17 '15 at 09:40
  • What if I use redis in an app and the following execution of the program depends on the KEYS (or SCAN) result. I mean the program needs all the data from this operation to continue calculation. I think this is the case where KEYS is more preferable than SCAN, isn't it? – Dany Sep 02 '16 at 13:55
  • @Dany I don't think so. Most programming languages have frameworks or libraries to handle this scenarios using thread synchronization, or even process synchronization with mutexes and other approaches :D – Matías Fidemraizer Sep 02 '16 at 16:53
11

The answer is in the SCAN documentation

These commands allow for incremental iteration, returning only a small number of elements per call, they can be used in production without the downside of commands like KEYS or SMEMBERS that may block the server for a long time (even several seconds) when called against big collections of keys or elements.

So ask for small chunks of data rather than getting whole of it

Also as Matías Fidemraizer pointed out, Redis is single threaded and KEYS is a blocking call thus blocking any incoming requests for operation until execution of KEYS is done.

Whether your data is small or not, it never hurts to apply best practices.

Jake
  • 7,565
  • 6
  • 55
  • 68
Basit Anwer
  • 6,742
  • 7
  • 45
  • 88
  • 9
    Clarification - all commands in Redis are blocking on the server since it's single-threaded. Keys just has the potential to block for a long time trying to get all the keys in one operation rather than sending small amounts of keys back in several operations. – Mani Gandham Jul 05 '16 at 02:06
8

  1. There is no performance difference between KEYS and SCAN other than pagination (count) where the amount bytes transferred (IO) from redis to client will be controlled in pagination.

  2. The count option it self has its own specification where sometimes you will not get data, but still scan cursor is on, so will get data in the next iterations. So the count option should be reasonable amount say 200 to something max to avoid multiple round trip time. I think this value depends on total number of keys in your db.

  3. There is no point/difference when we use SCAN within LUA compare to KEYS, though there is no IO involved, still both are blocking other calls till entire big collection get iterated. I haven't tried this, my guess it is.

Kanagavelu Sugumar
  • 18,766
  • 20
  • 94
  • 101