We are using Apache Unomi v1.5.7. We are doing load testing to check how many events it can process. We tried it with and without the custom plugin (written by us). don't look promising to us.
Resources: 8 CPU / 16 GB RAM
Infra: Deployed on AWS through K8s
Elastic Search Cluster: 6 Nodes (3 Master and 3 Data)
We ran the concurrent threads equal to the no of concurrent users in the below table and repeated this process 60 times with an interval of 1 second.
Here are the results: w/o Custom Plugin:
| No. of concurrent users | Throughput | Latency(ms) - 90 percentile | Latency(ms) - 95 percentile | Latency(ms) - 99 percentile |
|---|---|---|---|---|
| 50 | 44/sec | 200 | 306 | 1502 |
| 100 | 82/sec | 595 | 882 | 1177 |
| 200 | 135/sec | 609 | 825 | 2562 |
With Custom Plugin:
| No. of concurrent users | Throughput | Latency(ms) - 90 percentile | Latency(ms) - 95 percentile | Latency(ms) - 99 percentile |
|---|---|---|---|---|
| 50 | 33/sec | 688 | 1094 | 2538 |
| 100 | 51/sec | 1315 | 1572 | 3236 |
| 200 | 52/sec | 3785 | 4615 | 5997 |
Note: I can't share the exact elastic search mapping for obvious reasons but to give some idea, We have 2 array fields that are of nested type and ~10 more fields in the profile.
This test contains the cases where new profile are created, we haven't tested the case when existing profiles are updated.
What are the options that we can explore to improve the Unomi performance?
We have added enough resources to ensure it's not a resource issue.
We explored the properties defined in this file but properties like the number of threads don't seem to be affected at all.
