Parallel Programming - Python

Question

I have recently heard some things about parallel programming and I know basically nothing about it but will read up on it. But as a start, is it possible to run for instance this code:

for i in range(1000):
    print('Process1:', i)
    

for j in range(1000):
    print('Process2:', j)

in parallel? This case is just a toy example but it would help the understanding of the "potential" of parallel programming.

And, if that code can be ran in parallell, will it print out output from both loops in parallell in the following manner:

Process1: 0

Process2: 0

Process1: 1

.

or what?

[`threading`](https://docs.python.org/3/library/threading.html), [`multiprocessing`](https://docs.python.org/3/library/multiprocessing.html), [`concurrent.futures`](https://docs.python.org/3/library/concurrent.futures.html), [`asyncio`](https://docs.python.org/3/library/asyncio.html). Choose your warrior. I recommend you to read next documentation [Concurrent Execution](https://docs.python.org/3/library/concurrency.html). — Olvin Roght, Jan 15 '22 at 21:13

Another Noob · Answer 1 · 2022-01-15T22:12:12.027

Short anser is yes, and how the output will look depends on which method you use. For example if you want to use concurrent.futures, you can print the output as it is performed inside the function, and order will be scrambled. If you instead want to access return values, you can chose if you want to access them as they are completed, in whichever order they happen to finish, or you can use the "map" function to retrieve them in the expected order.

import concurrent.futures


def test_function(arguments: tuple):
    test_value, function = arguments
    """Print the test value 1000 times"""
    for i in range(0, 1000):
        print(f"Function {function}, {test_value}, iteration {i}")

    return test_value


def main():
    """Main function"""
    
    # Context manager for parallel tasks
    with concurrent.futures.ThreadPoolExecutor() as executor:

        # Submit example. Executes the function calls asynchronously
        result = [executor.submit(test_function, (i, "submit")) for i in range(1, 21)]

        # Map example. 
        # Takes an iterable as argument that will execute the function once for each item
        result_2 = executor.map(test_function, [(i, "map") for i in range(1, 21)])

    for future in concurrent.futures.as_completed(result):
        print(f"Submit: Process {future.result()} completed")

    for future in result_2:
        print(f"Map: Process {future} completed")


if __name__ == '__main__':
    main()

score 1 · Accepted Answer · answered Jan 15 '22 at 22:06

Since you are a beginner, the same with threading below: (Note that the output is not controlled in any way and might mix up!)

import threading
def f1():
    for i in range(1000):
        print('Process1:', i)
    
def f2():
    for j in range(1000):
        print('Process2:', j)

t1=threading.Thread(target=f1)
t2=threading.Thread(target=f2)
t1.start()
t2.start()

Parallel Programming - Python

2 Answers2

Linked