3

Is there a way, using which we can predict the N/W latency between two different operators in Apache Apex.

In a situation where the data that is going to be used in an operator, resides on a different machine altogether; this will add to the latency of the operator.

How can this be calculated?

Community
  • 1
  • 1
frewper
  • 1,385
  • 6
  • 18
  • 44

2 Answers2

2

You may want to look at a couple of blog articles:

https://www.datatorrent.com/blog/latency-calculation-in-apache-apex/ https://www.datatorrent.com/blog/blog-apex-performance-benchmark/

Sanjay
  • 141
  • 1
2

Assuming your tuples are strings and that the clocks on your cluster nodes are synchronized, you can append a timestamp to each tuple in the sending operator. Then, in the receiving operator, you can strip out the timestamp and compare it to the current time. You can, of course, suitably adapt this approach for other types. If averaged over a suitably large number of tuples, it should give you a good approximation of the network latency.

Permutation
  • 171
  • 1
  • 4
  • Isnt there any API that is provided by Apex that I can use. I am assuming this is pretty standard thing, hence should be already present ? – frewper Feb 22 '17 at 13:10