How to make my code to not run sequentially in Python using multithreading

Question

I created simple code which sum the array using Python and multithreading, but I know that It works sequentially. My question is, how can I change my code to not work sequentially using multithreading ? The code for sum:

import threading
matrix = [6,5,4,3,2,1,0]
def sum(n, **totalSum):
    sumThreads['sumThreads'] += matrix[n]

sumThreads = {"sumThreads":0}
for i in range(len(matrix)):
    t = threading.Thread(target=sum, args=(i,), kwargs=sumThreads)
    t.start()
    t.join()
print("Suma: ",sumThreads['sumThreads'])

The code for finding minimum:

import threading
matrix = [6,5,4,3,2,1,0]
def min(n, **total):
    if matrix[n-1] <= matrix[n]:
        if (matrix[n] < minThreads['minThreads']):
            minThreads['minThreads'] = matrix[n]
    else:
        if (matrix[n] < minThreads['minThreads']):
            minThreads['minThreads'] = matrix[n]


minThreads = {"minThreads": matrix[1]}
for i in range(len(matrix)):
    t = threading.Thread(target=min, args=(i,), kwargs=minThreads)
    t.start()
    t.join()
print("Minimum: ",minThreads['minThreads'])

I tried to achive it using multiprocessing like so:

if __name__ == '__main__':
    import multiprocessing
    my_input = [6,5,4,3,2,1,0]
    # Pool over all CPUs
    print(sum(multiprocessing.Pool().map(int, my_input)))
    print(min(multiprocessing.Pool().map(int, my_input)))

but I need to use multithreading. Any help would be greatly appreciated!

Does this answer your question? [What is the global interpreter lock (GIL) in CPython?](https://stackoverflow.com/questions/1294382/what-is-the-global-interpreter-lock-gil-in-cpython) — Askold Ilvento, Nov 18 '21 at 14:58

Corralien · Accepted Answer · 2021-11-18T19:29:03.723

1

You cannot share your list my_input between processes. A good strategy is to split your data into small chunks then compute partial data and finally process all partial results:

You code could be:

import multiprocessing as mp
import time
import random

CHUNKSIZE = 3

def partial_sum(l):
    # Include a latency for demo
    time.sleep(random.random())
    s = sum(l)
    print(f"Sum of {str(l):<15}: {s:>3}")
    return s

if __name__ == '__main__':
    matrix = [6,5,4,3,2,1,0]
    chunks = (matrix[i:i+CHUNKSIZE]
                  for i in range(0, len(matrix), CHUNKSIZE))

    with mp.Pool(mp.cpu_count()) as pool:
        global_sum = sum(pool.map(partial_sum, chunks))
    print('---------------------------')
    print(f"Global total: {global_sum}")

Output:

Sum of [3, 2, 1]      :   6
Sum of [6, 5, 4]      :  15
Sum of [0]            :   0
---------------------------
Global total: 21

Update

Find the minimum:

import multiprocessing as mp

CHUNKSIZE = 3

def local_min(l):
    m = min(l)
    print(f"Local minimum of {l} is {m}")
    return m

if __name__ == '__main__':
    matrix = [6,5,4,3,2,1,0]
    chunks = (matrix[i:i+CHUNKSIZE]
                  for i in range(0, len(matrix), CHUNKSIZE))

    with mp.Pool(mp.cpu_count()) as pool:
        l = pool.map(local_min, chunks)
        m = min(l)
    print('--------------------------------')
    print(f"Global minimum of {l} is {m}")

Output:

Local minimum of [6, 5, 4] is 4
Local minimum of [3, 2, 1] is 1
Local minimum of [0] is 0
--------------------------------
Global minimum of [4, 1, 0] is 0

edited Nov 18 '21 at 19:29

answered Nov 18 '21 at 13:56

Corralien

109,409
8
28
52

Sorry I think my answer is out of topic. I can remove it if you prefer – Corralien Nov 18 '21 at 14:01
It's a little bit out of topic, but I have a question. How would your example work If I wanted to sum the array using multiprocessing ? Like in my example when using array named matrix. If you could change your code to work with the array I would be very grateful. Maybe that is the way instead of using multithreading. – neekitit Nov 18 '21 at 18:52
`matrix` is not defined in your code. – Corralien Nov 18 '21 at 18:54
Yeah I forgot to add it. Now It is defined. – neekitit Nov 18 '21 at 18:58
I think I will mark your post as the answer, but if you have time and willingness I will have one more question to you. I added my code for finding minimum in array using multithreading. Could you do the same for this code as you done for sum ? I mean changing It from multithreading to multiprocessing. That two examples would really help me with other exercises. – neekitit Nov 18 '21 at 19:14
I updated my answer. I hope is what you expect? – Corralien Nov 18 '21 at 19:29
1

Yeah It's fantastic. Thanks! – neekitit Nov 18 '21 at 19:31

How to make my code to not run sequentially in Python using multithreading

1 Answers1