1

I notice strange behavior in the performance of Flink Gelly (PageRank algorithm on a graph of approximately 17,000,000 vertices and 136,000,000 edges).

While setting taskmanager.heap.mb: 125000 results to a runtime of ~110 sec, taskmanager.heap.mb: 10000 reduces the runtime by more than 50%.

I understand that big JVMs take a lot of time to initialize and garbage collection is slower too, but are those reasons sufficient for such a big difference in performance?

rawrintheclouds
  • 89
  • 8
  • More information required. Did you ensure that the page caches were in the same state (either warmed up or cold)? Did you compare IO loads? How much ram does your system have? Try `perf` or `time -v`. – the8472 Feb 01 '18 at 19:30
  • Yes, the caches were cold in both cases. All machines in my cluster have 252 GB of RAM. I/O load seems to consist of a more reads in the smaller JVM case. – rawrintheclouds Feb 02 '18 at 09:08

0 Answers0