21

I played around with flask microframework, and wanted to cache some stats in redis. Let's say I have this dict:

mydict = {}
mydict["test"] = "test11"

I saved it to redis with

redis.hmset("test:key", mydict)

However after restore

stored = redis.hgetall("test:key")
print(str(stored))

I see weird {b'test': b'test11'} so stored.get("test") gives me None

mydict str method result looks fine {'test': 'test11'}. So, why this binary marker added to restored data? I also checked in redis-cli and don't see explicit b markers there. Something wrong with hgetall?

Tommi
  • 3,199
  • 1
  • 24
  • 38
  • 4
    If someone interested, here is answer from developer: https://github.com/andymccurdy/redis-py/issues/463#issuecomment-41229918 – Tommi Apr 24 '14 at 08:29

3 Answers3

31

This is the intended behavior. By default, strings coming out of Redis don't get decoded. You have a couple options:

  • Decode the data yourself.
  • Create a client instance with the decode_responses argument, e.g., StrictRedis(decode_responses=True). This will decode all strings that come from Redis based on the charset argument (which defaults to utf-8). Only do this is you're sure every response from Redis has string data that you want decoded to utf-8. If you're using the same client instance to get binary data such as a pickled object, you shouldn't use this option. In that case, I'd suggest using a separate client instance for the binary data.

Source: https://github.com/andymccurdy/redis-py/issues/463#issuecomment-41229918

fnkr
  • 9,428
  • 6
  • 54
  • 61
  • 3
    This argument is undocumented and it doesn't seems to work: `python3` `>>> import redis` `>>> POOL = redis.ConnectionPool(host='localhost', port=6379, db=0)` `>>> datastore = redis.StrictRedis(connection_pool=POOL, decode_responses=True)` `>>> mydict = {}` `>>> mydict["test"] = "test11"` `>>> datastore.hmset("test:key", mydict)` `True` `>>> stored = datastore.hgetall("test:key")` `>>> stored` `{b'test': b'test11'}` – mezzomondo Jul 20 '15 at 12:59
11
POOL = redis.ConnectionPool(host='localhost', **decode_responses=True**, port=6379, db=0)
datastore = redis.StrictRedis(connection_pool=POOL)

if you use the ConnectionPool, you should move the decode_responses=True to the constructor of ConnectionPool.

Haifeng Zhang
  • 30,077
  • 19
  • 81
  • 125
mark gu
  • 111
  • 1
  • 2
6

Here's a simple routine to perform the byte decoding for the redis dict returned by HGETALL 

def decode_redis(src):
    if isinstance(src, list):
        rv = list()
        for key in src:
            rv.append(decode_redis(key))
        return rv
    elif isinstance(src, dict):
        rv = dict()
        for key in src:
            rv[key.decode()] = decode_redis(src[key])
        return rv
    elif isinstance(src, bytes):
        return src.decode()
    else:
        raise Exception("type not handled: " +type(src))

... and to use, just pass it the redis dict ...

obj = decode_redis(redis_connection.hgetall(key))
Darren Smith
  • 2,261
  • 16
  • 16