0

I have the following code for parallel computing of finding shortest path length between nodes in a graph.

For some reason, pool.map() works, but pool.apply() doesn't.

Could you help me trouble shooting what the problem is?

import multiprocessing as mp
import time
import networkx as nx


G = nx.fast_gnp_random_graph(2000, 0.001)

# Single core
current_time = time.time()
lengths = [nx.single_source_shortest_path_length(G, node) for node in G.nodes]
time_consumed = time.time() - current_time
print('single core time: %s' % time_consumed)


# pool map

def spl(node):
    return nx.single_source_shortest_path_length(G, node)

pool = mp.Pool(mp.cpu_count())
current_time = time.time()
results = pool.map(spl, G.nodes)
time_consumed = time.time() - current_time
pool.close()
print('pool map %s core time: %s' % (mp.cpu_count(), time_consumed))


# Pool apply
pool = mp.Pool(mp.cpu_count())
current_time = time.time()
results = [pool.apply(spl, args=(node,)) for node in G.nodes]
time_consumed = time.time() - current_time
pool.close()
print('pool apply %s core time: %s' % (mp.cpu_count(), time_consumed))

Output:

single core time: 25.18528151512146
pool map 12 core time: 5.043155670166016
pool apply 12 core time: 29.906842470169067
Ronnie Z
  • 31
  • 3
  • what do you mean by `apply` *doesn't work*? Is it giving different results, or are you referring to the run-time? – Tomerikoo Jan 24 '20 at 11:23
  • 2
    That should help https://stackoverflow.com/questions/8533318/multiprocessing-pool-when-to-use-apply-apply-async-or-map – Harion Jan 24 '20 at 11:27
  • @Tomerikoo Yes, the run-time is not faster with pool.apply(). I will read the similar question you recommended. Thanks for your reply! – Ronnie Z Jan 29 '20 at 13:25

0 Answers0