Unable to run three threads concurrently after performing thread synchronization through semaphore

Question

I am able to get sequential output for below multithreading code using semaphore with semaphore value set to 1. However instead of 3 processes running concurrently now there is just one thread running at a time which is expected. Is there a way where i can run three threads concurrently and also get sequential output as well?

from datetime import datetime
import getpass
import os
import sys
import time
import re
import json
from random import random
import threading
from io import StringIO
from time import gmtime, strftime
from pprint import pprint
from threading import *
screen_lock = Semaphore(value=1)

#------------------------------------------------------------------------------
def config_worker(port):
    if port == 'a':
       screen_lock.acquire()
       print('PASS : Tunnel1 is UP from  router 1')
       print('PASS : Tunnel2 is UP from  router 1')
       print('PASS : Tunnel3 is UP from  router 1')
       screen_lock.release()
       
    if port == 'b':
       screen_lock.acquire()
       print('PASS : Tunnel1 is UP from  router 2')
       print('PASS : Tunnel2 is UP from  router 2')
       print('PASS : Tunnel3 is UP from  router 2')
       screen_lock.release()
       
    if port == 'c':
       screen_lock.acquire()
       print('PASS : Tunnel1 is UP from  router 3')
       print('PASS : Tunnel2 is UP from  router 3')
       print('PASS : Tunnel3 is UP from  router 3')
       screen_lock.release()
    return

def connect():

    config_threads_list = []
    devices = ['a','b','c']
    for ports in devices:
        port = ports
        print ('Creating thread for: ', ports)
        config_threads_list.append(threading.Thread(target=config_worker, args=(port)))

    print ('\n---- Begin get config threading ----\n')
    for config_thread in config_threads_list:
        config_thread.start()

    for config_thread in config_threads_list:
        config_thread.join()



connect()

output with semaphore. output is correct but only one thread is running at a time which is expected. How to run all threads and print sequential output as well?

---- Begin get config threading ----

PASS : Tunnel1 is UP from  router 1
PASS : Tunnel2 is UP from  router 1
PASS : Tunnel3 is UP from  router 1
PASS : Tunnel1 is UP from  router 2
PASS : Tunnel2 is UP from  router 2
PASS : Tunnel3 is UP from  router 2
PASS : Tunnel1 is UP from  router 3
PASS : Tunnel2 is UP from  router 3
PASS : Tunnel3 is UP from  router 3

guys any suggestions will be helpful. If it can be done please let me know the same — Diwakar SHARMA, Jan 24 '21 at 10:56
Threads don’t actually run ‘concurrently’ in the sense of simultaneously - and basically threads only switch when there’s a reason - based around i/o usually, but I believe it’s possibe to use e.g. time.sleep(0) to get a thread to yield so Python runs another thread if there’s one that can be run. If you remove the locking and add `time.sleep(0)` after each print statement you may see this happening. But aiming to control thread switching is contradictory - if you think you need to completely control the sequence threads execute then you probably shouldn’t use threads, write sequential. — DisappointedByUnaccountableMod, Jan 24 '21 at 11:51
Correction - don’t use 0, use e.g. `time.sleep(0.0001’)` see https://stackoverflow.com/questions/787803/how-does-a-threading-thread-yield-the-rest-of-its-quantum-in-python — DisappointedByUnaccountableMod, Jan 24 '21 at 12:08

DisappointedByUnaccountableMod · Accepted Answer · 2021-01-25T16:35:38.307

You are running all threads - you can tell this because all the output is produced.

You always will get the statements in blocks of three because of the semaphore.

If you want something a bit more random sequence, try adding a sleep (I also added a print to show which port thread is starting):

from datetime import datetime
import getpass
import os
import sys
import time
import re
import json
from random import random
import threading
from io import StringIO
from time import gmtime, strftime
from pprint import pprint
from threading import *
screen_lock = Semaphore(value=1)

#------------------------------------------------------------------------------
def config_worker(port):
    print( f"Starting thread {port}" )   # ADDED
    time.sleep(0.0001)                   # ADDED
    if port == 'a':
       screen_lock.acquire()
       print('PASS : Tunnel1 is UP from  router 1')
       print('PASS : Tunnel2 is UP from  router 1')
       print('PASS : Tunnel3 is UP from  router 1')
       screen_lock.release()
       
    if port == 'b':
       screen_lock.acquire()
       print('PASS : Tunnel1 is UP from  router 2')
       print('PASS : Tunnel2 is UP from  router 2')
       print('PASS : Tunnel3 is UP from  router 2')
       screen_lock.release()
       
    if port == 'c':
       screen_lock.acquire()
       print('PASS : Tunnel1 is UP from  router 3')
       print('PASS : Tunnel2 is UP from  router 3')
       print('PASS : Tunnel3 is UP from  router 3')
       screen_lock.release()
    return

def connect():

    config_threads_list = []
    devices = ['a','b','c']
    for ports in devices:
        port = ports
        print ('Creating thread for: ', ports)
        config_threads_list.append(threading.Thread(target=config_worker, args=(port)))

    print ('\n---- Begin get config threading ----\n')
    for config_thread in config_threads_list:
        config_thread.start()

    for config_thread in config_threads_list:
        config_thread.join()



connect()

Now when you run this code you can get different sequences, try it a few times:

FIRST RUN:

Creating thread for:  a
Creating thread for:  b
Creating thread for:  c

---- Begin get config threading ----

Starting thread a
Starting thread b
Starting thread c
PASS : Tunnel1 is UP from  router 3
PASS : Tunnel2 is UP from  router 3
PASS : Tunnel3 is UP from  router 3
PASS : Tunnel1 is UP from  router 1
PASS : Tunnel2 is UP from  router 1
PASS : Tunnel3 is UP from  router 1
PASS : Tunnel1 is UP from  router 2
PASS : Tunnel2 is UP from  router 2
PASS : Tunnel3 is UP from  router 2

SECOND RUN:

Creating thread for:  a
Creating thread for:  b
Creating thread for:  c

---- Begin get config threading ----

Starting thread a
Starting thread b
Starting thread c
PASS : Tunnel1 is UP from  router 2
PASS : Tunnel2 is UP from  router 2
PASS : Tunnel3 is UP from  router 2
PASS : Tunnel1 is UP from  router 1
PASS : Tunnel2 is UP from  router 1
PASS : Tunnel3 is UP from  router 1
PASS : Tunnel1 is UP from  router 3
PASS : Tunnel2 is UP from  router 3
PASS : Tunnel3 is UP from  router 3

Or if you comment out the screen_lock.acquire()/screen_lock.release() lines then you get a more mixed up sequence (in this case I left the sleep(0.0001) in but the sequence also varies without the sleep, presumably because of thread switching on I/O of the print):

Creating thread for:  a
Creating thread for:  b
Creating thread for:  c

---- Begin get config threading ----

Starting thread a
Starting thread b
Starting thread c
PASS : Tunnel1 is UP from  router 3
PASS : Tunnel1 is UP from  router 1
PASS : Tunnel2 is UP from  router 3
PASS : Tunnel2 is UP from  router 1
PASS : Tunnel3 is UP from  router 3
PASS : Tunnel3 is UP from  router 1
PASS : Tunnel1 is UP from  router 2
PASS : Tunnel2 is UP from  router 2
PASS : Tunnel3 is UP from  router 2

score 0 · Answer 2 · answered Jan 24 '21 at 11:29

0

Most Probably GIL Lock is the problem. Try multiprocessing.pool

from multiprocessing import Pool

def connect():
    po = Pool(3)
    res = po.map(config_worker, devices)

answered Jan 24 '21 at 11:29

Saket Mittal

3,726
3
29
49

Don’t believe there is a ‘problem’ it’s just how Python threads work. – DisappointedByUnaccountableMod Jan 24 '21 at 12:09
hi saket sorry multiprocessiing is causing my windows pc to hang , do you have a bettern solution , i can't receive any output after adding your code – Diwakar SHARMA Jan 25 '21 at 14:03
On Windows that code will almost inevitability lock you up - you need to protect the code to be executed as the 'main' thread using `if __name__=='__main__':` see docs https://docs.python.org/3/library/multiprocessing.html where all examples use this. Multiprocessing is a very different beast from threading. – DisappointedByUnaccountableMod Jan 25 '21 at 16:35

Unable to run three threads concurrently after performing thread synchronization through semaphore

2 Answers2