Multiprocessing in Python
CPython, the most common implementation of the Python Interpreter, does not have native support for multiprocessing without using the multiprocessing module of python. This is due to the Global Interpreter Lock (GIL) introduced as python memory is not thread safe, and thus even the threading module does not truly implement parallel code. The multiprocessing module on the other hand starts entirely new instances of the Python interpreter, and as they no longer share a GIL can truly be run in parallel across multiple cores.
Therefore if you wish to run a python program that takes advantage of multiple cores your must break up your program manually into sub-processes that each can be run on a separate instance of the interpreter. Do be careful though, as the gains from taking advantage of multiple cores in this way may cost more in overhead than is gained by running across cores, but this is very project dependant, so you should check this for yourself.
Sources:
Multicore Python: A tough, worthy, and reachable goal
Multi-Core and Distributed Programming in Python
Multiprocessing vs Threading Python
Possible Solution to your problem
Now bear with me for a while here, as there are a couple of mathematical steps, but I do believe that there is a way for your code to be parallelised.
First take example outputs for different values of nums, to see what the output of the code would be:
nums | 1 | 2 | 3 | 4 | 5 | 6 |
-----------------------------------
out | 1 | 2 | 6 | 24 | 120 | 720 |
Now due to your code is set up you can see that code(nums) = code(nums - 1) + code(nums - 1) * (nums- 1), so if we can find a way to get from code(nums - 1) to code(nums) without actually needing to know the value of code(nums - 1) then we can paralellise the code.
If we now consider the difference of going from code(nums - 1) to code(nums) in the above table we get the following table of their differences:
nums | 1 | 2 | 3 | 4 | 5 | 6 |
-----------------------------------
diff | | 1 | 4 | 18 | 96 | 600 |
Now these differences might not seem to form a pattern, but actually with a bit of a logical leap, you can see that the difference between code(nums) and code(nums - 1) is (nums - 1) * (nums - 1)! (where ! is the mathematical factorial function). Thus we see that your code method can equivalently be written as:
from math import factorial
def code(nums):
num = 1
for i in range(1, nums):
num += i * factorial(i)
return num
Now we are somewhere that we can parallelise your code from. We'll use the multiprocessing module with a pool of 8 processes (to match your 8 cores), and map a section of the work needed onto each core, allow them to compute their desired value and then total up the results, then simply adding 1 to the result to account for the initial value of num.
from math import factorial
from multiprocessing import Pool, freeze_support
NUM_CORES = 8
def foo(lower, upper):
sum = 0
for i in range(lower, upper):
sum += i * factorial(i)
return sum
def code(nums):
# Build the list of arguments for the workers so that each gets 1/NUM_CORES of the work.
a, b = divmod(nums, NUM_CORES)
arguments = [(1,a)]
for c in range(2, NUM_CORES):
arguments.append(((c-1)*a, c*a))
arguments.append(((NUM_CORES - 1)*a, NUM_CORES*a + b))
print(arguments)
with Pool(processes=8) as pool:
results = pool.starmap(foo, arguments)
print(1 + sum(results))
if __name__ == '__main__':
freeze_support()
code(100)
The freeze_support call in the __main__ block at the bottom is needed so that the multiprocessing is performed correctly in the code method.
