In my application there are three erlang processes which run periodically at an interval of 10ms, 1s and 5s, let's call them as process A, B and C respectively; and there are two c-nodes, X and Y which perform some low level hardware access.
The c-nodes has a fixed pool of linux thread to serve the queries received from the erlang process simultaneously. One erlang process can send only one query to a c-node as the calls are blocking, but the c-nodes can receive multiple calls from different erlang process and serve them simultaneously through it's fixed pool of threads.
In my application process A and C are using the service of c-node X, and process is B using the service of c-node Y.
We observed that the execution time of all the three erlang processes are varying drastically over time, i.e. to start with process A, B and C is completing their task in 3ms, 100ms and 250ms respectively, but over a period of an hour it is increasing to 200ms, 600ms, 1300ms, then again it is coming down slowly to 3ms, 100ms and 250ms, and as we continue to run we are seeing this pattern repetitively.
I have also implemented the same erlang process logic in C application, there I do not see this issues.
What could be the possible explanation behind this slow down pattern, and what would be the right way to benchmark or profile this behaviour?