Redis: intersection of sorted sets by score

Question

I want to store location triples in a Redis datastore, but want to make them searchable as well. This would make it possible to do range queries, like 'give me al points with 1 < x 3 and y > 2'. Therefore, I use a combination of sorted sets.

Each triple is saved in Redis like so (e.g. location A with x = 1, y = 2, z = 3):

hset /locations/A x "1"
hset /locations/A y "2"
hset /locations/A z "3"
hset /locations/A payload "{ ...some json payload... }"
zadd /locations:x 1 locations/A
zadd /locations:y 2 locations/A
zadd /locations:z 3 locations/A

This way, I can easily find all locations (or paths to locations) with e.g. an x value between 4 and 5:

zrangebyscore /locations:x 4 5

Or all locations with e.g. an y value between 1 and 3:

zrangebyscore /locations:x 1 3

A problem arises when I try to match all locations with an x value between 4 and 5 AND an y value between 1 and 3, because then I have to do two queries to Redis, subsequently comparing the values with Javascript in NodeJS, which could be very time-consuming when a lot of locations are defined. Did anyone encounter such a problem?

I experimented with zinterstore and zunionstore, but haven't found a satisfactory solution yet. I considered storing the zrangebyscores into a temporary set and then doing a zinterstore, but didn't find a Redis command to store the output of zrangebyscore directly to Redis (in the same command).

Answer 1

I'd like to ignore the use case (locations paths/distances) itself because there are multiple proven ways to address this challenge, also with Redis (search for geospatial and ye shall find), and focus instead on the technique.

So, assuming you're going to invent and implement your own geo logic, the most effective way (short of modifying Redis' code) to address the challenges involved in scoring/ranging/intersecting/... these Sorted Sets will be inside of a Lua script. This is what you may buzzwordly call "Data Gravity" - the processor is close to the data so accessing and manipulating the data is fastest and requires no network.

Inside such scripts you can, for example, store the results of ZRANGEBYSCORE in local variables, do whatever you need to do there and with them, and reply with the end result to the (Node.js) client.

Answer 2

do it, do your two queries, Redis is fast and doesnt care. Then javascript Simplest code for array intersection in javascript to find the values that exist in both.