In Redis mode, the data sent is different from the received data

Question

In Redis mode, the sent data is different from the received data. The same problem with memcached. Here is an example of the code for Redis:

import pickle    
import redis

REDIS = {
    'host': 'localhost',
    'port': 6379,
    'db': 0
}

IGNITE = {
    'host': 'localhost',
    'port': 11211,
    'db': 0
}

def test_connection(redis_connection):
    d = {
        'a': 1,
        'b': 'AS213dfdsfфывфывфывфа',
        'c': None,
    }
    pickle_dumps = pickle.dumps(d)
    print(pickle_dumps)
    redis_connection.set('foo', pickle_dumps)
    print(redis_connection.get('foo'))
    print('-----------------------')


test_connection(redis.StrictRedis(**IGNITE))
test_connection(redis.StrictRedis(**REDIS))

That's how I run the Ignite:

docker run -p 11211:11211 -it -e "CONFIG_URI=https://raw.githubusercontent.com/apache/ignite/master/examples/config/redis/example-redis.xml" apacheignite/ignite

Output:

b'\x80\x03}q\x00(X\x01\x00\x00\x00aq\x01K\x01X\x01\x00\x00\x00bq\x02X \x00\x00\x00AS213dfdsf\xd1\x84\xd1\x8b\xd0\xb2\xd1\x84\xd1\x8b\xd0\xb2\xd1\x84\xd1\x8b\xd0\xb2\xd1\x84\xd0\xb0q\x03X\x01\x00\x00\x00cq\x04Nu.'
b'\xef\xbf\xbd\x03}q\x00(X\x01\x00\x00\x00aq\x01K\x01X\x01\x00\x00\x00bq\x02X \x00\x00\x00AS213dfdsf\xd1\x84\xd1\x8b\xd0\xb2\xd1\x84\xd1\x8b\xd0\xb2\xd1\x84\xd1\x8b\xd0\xb2\xd1\x84\xd0\xb0q\x03X\x01\x00\x00\x00cq\x04Nu.'
-----------------------
b'\x80\x03}q\x00(X\x01\x00\x00\x00aq\x01K\x01X\x01\x00\x00\x00bq\x02X \x00\x00\x00AS213dfdsf\xd1\x84\xd1\x8b\xd0\xb2\xd1\x84\xd1\x8b\xd0\xb2\xd1\x84\xd1\x8b\xd0\xb2\xd1\x84\xd0\xb0q\x03X\x01\x00\x00\x00cq\x04Nu.'
b'\x80\x03}q\x00(X\x01\x00\x00\x00aq\x01K\x01X\x01\x00\x00\x00bq\x02X \x00\x00\x00AS213dfdsf\xd1\x84\xd1\x8b\xd0\xb2\xd1\x84\xd1\x8b\xd0\xb2\xd1\x84\xd1\x8b\xd0\xb2\xd1\x84\xd0\xb0q\x03X\x01\x00\x00\x00cq\x04Nu.'
-----------------------

How can I fix this?

score 2 · Answer 1 · answered Apr 09 '18 at 13:18

I can see that \x80 is replaced by \xef\xbf\xbd.

The latter is Unicode representation of Replacement character

I expect this is the result of writing \x80 (a control character) into Unicode stream inside Ignite. I could find a question about Pickle with a similar problem.

It also advices against using Pickle in network environment. Why don't you use e.g. JSON?

In Redis mode, the data sent is different from the received data

1 Answers1