We send messages from ArcSight to StreamSets pipeline using Kafka. We are experiencing trouble parsing the messages from Kafka in the pipeline. The data sent from ArcSight is sometimes partitioned into chunks which means that a huge script will be cut into small chunks and separated into multiple messages. When this happens the Kafka Consumer used in the StreamSets pipeline cant parse the data correctly.
any help?
