My goal is to get ArcSight CEF messages from ArcSight Transformation Hub into Elasticsearch in the parsed form. Hoever, none of the input options available from Elastic Stack do not work:
Logstash ArcSight module does not work at all via SSL with Transformation Hub: Kafka Consumer Failed to load SSL keystore (Logstash ArcSight module) for any keystore type and path
Logstash with Kafka Plugin can indeed read data from ArcSight Transofrmation Hub using SSL, but it only reads approximately 50% of the data, so that Kafka queue lag always increases, no matter what kind of setting I try (I have also tried to have 9 Logstash instances, each reading 1 of 9 Transformation Hub / Kafka pipeline, but still get the same amount of events as single Logstash instance):
Filebeat also works with ArcSight Transformation Hub via SSL with client certificate authentication. However, it uses decode_cef processor. Without a modified filebeat pipeline, it is unable to parse some Windows Events and has maximum 5k EPS (I need 50k EPS). With a custom pipeline, it probably can parse everything, but almost no events are coming into the Elasticsearch (5000 events per hour instead of 50,000 events per second).
So the question is how to get data from ArcSight into Elasticsearch if none of 3 "standard" options offered by Elastic work?
