0

My Nifi processor implementation is generating a strange cache key and value, something like '\xac\xed\x00\x05t\x00\x06' is prepended to each key/ val. For example, inserting 'key':'val' generates "\xac\xed\x00\x05t\x00\x03key":"\xac\xed\x00\x05t\x00\x03val".

Nifi uses spring data in the background, issue described here:

Spring boot + Redis - Generating a strange key

Spring boot caching with redis,key have \xac\xed\x00\x05t\x00\x06

Nifi processor is just java code that overrides provided method.

I don't have spring config available to edit or spring data classes, that's all done within Nifi's implementation. All I have, and have to provide is serializer and deserializer:

void serialize(T value, OutputStream output) throws SerializationException, IOException;
T deserialize(byte[] input) throws DeserializationException, IOException;

For example getAndPutIfAbsent method looks like this:

cache.getAndPutIfAbsent(key, value, keySerializer, valueSerializer, valueDeserializer);

My byte serialization/ deserialization is simple:

 public static byte[] serialize(Object obj) throws IOException {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        ObjectOutputStream os = new ObjectOutputStream(out);
        os.writeObject(obj);
        return out.toByteArray();
    }

    public static Object deserialize(byte[] data) {
        if (data == null || data.length == 0) return null;
        try {
            ByteArrayInputStream in = new ByteArrayInputStream(data);
            ObjectInputStream is = new ObjectInputStream(in);
            return is.readObject();
        } catch (Exception e) {
            // log
        }
        return null;
    }

This generates keys/ vals with prepended value. Put and get do work, I can get the value that I previously put using the same serializer and deserializer, except in one case.

I have few caches and they all use same object serializer/ deserializer pasted above. As I mentioned everything works except one cache where I use scan method (paging through all keys). On key deserialization it throws: java invalid stream header: EFBFBDEF. I don't get it why other caches work but this one throws exception, they are all serializing/ deserializing the same.

My goal is to get keys/ vals without prepended chars and working consistently. I can not tinker with spring configs and classes. How to proceed?

luka_roves
  • 75
  • 5

1 Answers1

0

Using following de/serialization for strings fixed both issues:

value.getBytes(StandardCharsets.UTF_8)
new String(value, StandardCharsets.UTF_8);

Not sure why, but it works now.

luka_roves
  • 75
  • 5