64

I'm using the regular redis package in order to connect my Python code to my Redis server.

As part of my code I check if a string object is existed in my Redis server keys.

string = 'abcde'
if string in redis.keys():
  do something..

For some reasons, redis.keys() returns a list with bytes objects, such as [b'abcde'], while my string is, of course, a str object.

I already tried to set charset, encoding and decode_responses in my redis generator, but it did not help.

My goal is to insert the data as string ahead, and not iterate over the keys list and change each element to str() while checking it.

Thanks ahead

GMe
  • 1,091
  • 3
  • 13
  • 24
  • 1
    Possible duplicate of [Best way to convert string to bytes in Python 3?](http://stackoverflow.com/questions/7585435/best-way-to-convert-string-to-bytes-in-python-3) – Maurice Meyer May 17 '17 at 13:40
  • Also http://stackoverflow.com/questions/25745053 – Bemmu May 17 '17 at 13:41
  • Does this answer your question? [About char b prefix in Python3.4.1 client connect to redis](https://stackoverflow.com/questions/25745053/about-char-b-prefix-in-python3-4-1-client-connect-to-redis) – Muhammad Faizan Fareed Feb 28 '20 at 05:37

4 Answers4

119

You can configure the Redis client to automatically convert responses from bytes to strings using the decode_responses argument to the StrictRedis constructor:

r = redis.StrictRedis('localhost', 6379, charset="utf-8", decode_responses=True)

Make sure you are consistent with the charset option between clients.

Note

You would be better off using the EXISTS command and restructuring your code like:

string = 'abcde'
if redis.exists(string):
    do something..

The KEYS operation returns every key in your Redis database and will cause serious performance degradation in production. As a side effect you avoid having to deal with the binary to string conversion.

rwolst
  • 12,904
  • 16
  • 54
  • 75
Tague Griffith
  • 3,963
  • 2
  • 20
  • 24
  • 1
    Thanks. And what if I need to use a value of dictionary's key? how can I convert it automatically to string instead of bytes? – GMe May 17 '17 at 19:45
  • 1
    I updated the answer with information about decode_responses. – Tague Griffith May 17 '17 at 20:07
  • Thanks again. I tried this one with `redis.Redis` inside a connection pool and it wasn't worked well.. Is there any difference between `StrictRedis` and `redis.Redis` in the `decode_responses` argument? I found in the documentation that they both should be fine with this argument – GMe May 17 '17 at 20:14
  • Adding decode_responses=True only worked for me when I added it to ConnectionPool initialization rather than StrictRedis initialization. connection_pool = redis.ConnectionPool( host=REDIS_HOST, port=REDIS_PORT, password=REDIS_PASSWORD, decode_responses=True) redis_db = redis.StrictRedis( connection_pool=connection_pool, charset='utf-8') – Tilek Dec 20 '18 at 00:02
  • 3
    `charset=utf-8` is wrong for two reasons. First, the `charset` parameter has been deprecated and if you use it, you'll get a warning. It's been replaced with the `encoding` parameter. Second, the default for the `encoding` parameter is `utf-8`, so no need to set it at all. – mlissner Nov 11 '20 at 01:10
  • Thanks. I cached values into Redis before sent to gql endpoint. I needed to transparently convert my values from the string type to types described in my Graphql schema additionally before sending them to the endpoint. But in the case of ENUM Graphql type, I have not to do conversion to string type because it isn't the string data type it is enum. At the same time, it isn't correct to send byte type. It looks more stupid. I used your recommendation and solved my issue. – Orlov Const Sep 23 '21 at 11:37
  • Is there an option for Sentinel? – Oleg Yablokov Oct 25 '21 at 11:33
  • I tried `decode_responses=True` but I get this error `UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9c in position 1: invalid start byte` – Shayan Jun 11 '22 at 12:29
  • I can't find any mention of StrictRedis in the redis.py docs. – odigity Jan 30 '23 at 20:40
  • 1
    @odigity The redis.py API has changed since this was written. See https://stackoverflow.com/questions/19021765/redis-py-whats-the-difference-between-strictredis-and-redis for more information. – Tague Griffith Feb 03 '23 at 03:41
32

If you do not wish to iterate the list for decoding, set your redis connection to automagically perform the decode and you'll receive your required result. As follows in your connection string, please notice the decode_responses argument:

rdb = redis.StrictRedis(host="localhost", port=6379, db=0, decode_responses=True)

Happy Coding! :-) (revised 13 Nov 2019)

RandallShanePhD
  • 5,406
  • 2
  • 20
  • 30
  • 1
    This didn't work. BTW, "charset" is deprecated. Use "encoding" instead'. – zzxwill Nov 09 '19 at 10:15
  • 1
    The @zzxwill - mods made! – RandallShanePhD Nov 13 '19 at 19:38
  • this is cool! While the majority of the results I expect are strings, I have a small percentage of queries where I would like to override this setting and actually obtain bytes. Is there a way to obtain a "derived" connection from an existing one? E.g. `rdb2 = rdb.copy(decode_responses=False)` or similar? – Pierre D Jul 29 '20 at 23:03
  • answering my own "question in a comment"... We can get the current connection params from `kwold = rdb.connection_pool.connection_kwargs` and make a new connection after mods as `kwnew = {**kwold, **kwargs}` and `rdbnew = redis.StrictRedis(**kwnew)`. – Pierre D Jul 29 '20 at 23:44
  • Following up on my previous comment, I now just posted this idea [here](https://stackoverflow.com/q/63164404/758174). – Pierre D Jul 30 '20 at 00:34
7

Previous answers are correct in that the decode_responses connection parameter is needed to do this automatically, what they omit is that this needs to be set on the ConnectionPool instead if the connections are made using the pool. The below snippet will set the 'decode_responses' value on all clients to true, even if the client sets itself to False during connection

pool = redis.ConnectionPool(host=REDIS_HOST, port=REDIS_PORT, db=REDIS_DB, password=REDIS_PWD, decode_responses=True)
user1596707
  • 134
  • 2
  • 10
  • 1
    Do you have an example you can show in your answer? - You should [edit] it and include one so people know what you mean. – user3788685 Oct 23 '18 at 11:24
  • Hi, I took the liberty to remove part of your answer that's not related to the problem at hand, there's no problem at all in answering old questions. – brasofilo Oct 23 '18 at 12:03
  • Thanks! Adding decode_responses=True only worked for me when I added it to ConnectionPool initialization rather than StrictRedis initialization. connection_pool = redis.ConnectionPool( host=REDIS_HOST, port=REDIS_PORT, password=REDIS_PASSWORD, decode_responses=True) redis_db = redis.StrictRedis( connection_pool=connection_pool, charset='utf-8') – Tilek Dec 20 '18 at 00:03
6

One solution can be:

decode the redis key 

print(key)
#output is : b'some_key'

print(key.decode())
#output is : 'some_key'

Updated :

Pass dictionary object into RedisPost class then decoding individual item and storing them as a object. 

class RedisPost():
   def __init__(self, dic):
      for k,v in dic.items():
          if not isinstance(k,int):
             var = k.decode()
             setattr(self,var,v.decode())


my_dic = {'a':12, 'b':123}
obj = RedisPost(my_dic)
print(obj.a) # output will be 12
Muhammad Faizan Fareed
  • 3,298
  • 34
  • 30
  • The OP mentions that they don't want to iterate and check: "and not iterate over the keys list and change each element to str() while checking it" – thanos.a Dec 21 '21 at 07:03