Multiprocessing in Python: execute two functions at the exact same time

Question

I want to execute the following two functions at exactly the same time.

from multiprocessing import Process
import os
import datetime

def func_1(title):
    now = datetime.datetime.now()
    print "hello, world"
    print "Current second: %d" % now.second
    print "Current microsecond: %d" % now.microsecond

def func_2(name):
    func_1('function func_2')
    now = datetime.datetime.now()
    print "Bye, world"
    print "Current second: %d" % now.second
    print "Current microsecond: %d" % now.microsecond

if __name__ == '__main__':
    p = Process(target=func_2, args=('bob',))
    p.start()
    p.join()

And I am getting a difference in microseconds. Is there any way to execute both at the exact same time? Any help would be appreciated.

The *exact* same time is theoretically impossible - the question is how big your error margin is allowed to be. In your case I guess you want to start them at the same microsecond? — l4mpi, Nov 20 '12 at 14:20
I don't like answers like that, so it's just a comment, but I think that on systems popular on the planet Earth these days it is, unfortunately, not possible. — piokuc, Nov 20 '12 at 14:22

score 5 · Answer 1 · edited May 23 '17 at 11:51

This is (1) generally impossible (the "exact" part) and (2) not something that Python is good at. If you really need microsecond execution precision, use C or ASM, but an even closer way than COpython's answer would be busy-waiting in two different processes for an agreed start time:

from multiprocessing import Process
import os
import datetime
from time import time

def func_1(title):
    now = datetime.datetime.now()
    print "hello, world"
    print "Current second: %d" % now.second
    print "Current microsecond: %d" % now.microsecond

def func_2(name):
    now = datetime.datetime.now()
    print "Bye, world"
    print "Current second: %d" % now.second
    print "Current microsecond: %d" % now.microsecond

def start_f1(name):
    while time() < start_time: pass
    func_1(name)

def start_f2(name):
    while time() < start_time: pass
    func_2(name)        

if __name__ == '__main__':
    procs = []
    procs.append(Process(target=start_f2, args=('bob',)))
    procs.append(Process(target=start_f1, args=('sir',)))
    start_time = time() + 10
    map(lambda x: x.start(), procs)
    map(lambda x: x.join(), procs)

I've used this idea and I've gotten my processes to run consistently within 10ms of each other. Do you think that is the limit of this? any idea how to improve it even more? — clifgray, Jul 06 '18 at 19:10
@clifgray writing a C extension could improve this a bit, as could using Barrier instead of the busy wait (see the accepted answer; python3 only). There's also things you could do on the OS side, e.g. tuning the process scheduler. But if a 10ms difference is not good enough for your use case I doubt this will help, as any real work you do afterwards won't stay synchronized (at least not without your intervention). What good is it to start on the same microsecond when the actual actions you want to synchronize can still be apart by milliseconds or more? — l4mpi, Jul 09 '18 at 09:23

score 5 · Accepted Answer · answered Nov 20 '12 at 17:53

5

On the computer the following was written on, this code consistently prints out the same timestamps:

#! /usr/bin/env python3
from multiprocessing import Barrier, Lock, Process
from time import time
from datetime import datetime

def main():
    synchronizer = Barrier(2)
    serializer = Lock()
    Process(target=test, args=(synchronizer, serializer)).start()
    Process(target=test, args=(synchronizer, serializer)).start()

def test(synchronizer, serializer):
    synchronizer.wait()
    now = time()
    with serializer:
        print(datetime.fromtimestamp(now))

if __name__ == '__main__':
    main()

answered Nov 20 '12 at 17:53

Noctis Skytower

21,433
16
79
117

I am getting this error,`ImportError: cannot import name Barrier` and i am new to this threading i tried to import `Barrier` could not get it. i think that is not a inbuilt package – WEshruth Nov 22 '12 at 07:36
@Mr.Calm: Try using the latest release of Python (version 3.3) to use `Barrier`. – Noctis Skytower Nov 22 '12 at 13:15
A little late to the party but @NoctisSkytower why are you using the serializer here? I seem to get similar results just not using that at all. What is the motivation for it? – clifgray Jul 05 '18 at 17:29
@clifgray The `print` function does not construct a string before displaying it. There are several steps that are taken instead. The `serializer` is meant to ensure that all of those steps are atomic in nature. – Noctis Skytower Jul 05 '18 at 20:05

score 2 · Answer 3 · edited Aug 24 '16 at 13:55

2

I am not sure if this will execute at exactly the same time, but I think that it will get you closer.

from multiprocessing import Process
import os
import datetime


def func_1(title):
    now = datetime.datetime.now()
    print "hello, world"
    print "Current second: %d" % now.second
    print "Current microsecond: %d" % now.microsecond


def func_2(name):
    now = datetime.datetime.now()
    print "Bye, world"
    print "Current second: %d" % now.second
    print "Current microsecond: %d" % now.microsecond


if __name__ == '__main__':
    procs = []
    procs.append(Process(target=func_2, args=('bob',)))
    procs.append(Process(target=func_1, args=('sir',)))
    map(lambda x: x.start(), procs)
    map(lambda x: x.join(), procs)

edited Aug 24 '16 at 13:55

Noctis Skytower

21,433
16
79
117

answered Nov 20 '12 at 14:21

Cameron Sparr

3,925
2
22
31

I believe this is about as close as you can get in standard CPython without doing some sort of "real-time" extension. – apiguy Nov 20 '12 at 14:25
@COpython Sorry still there is 400ms diff. – WEshruth Nov 20 '12 at 14:25
@Mr.Calm, you could put both of your functions in a class, and get `now` in the constructor, then both functions would have the same timestamp... no other way to get it closer than that :-/ – Cameron Sparr Nov 20 '12 at 14:32

score 1 · Answer 4 · answered Nov 20 '12 at 15:57

1

CPythonis inherently single threaded (Google "Global Interpreter Lock"). To have even a theoretical chance you would need a multicore processor, but even then only an operating system operating at a very low level could do it and even then you would need special hardware.. What you are asking for is, in any practical sense, impossible.

answered Nov 20 '12 at 15:57

James Thiele

393
3
9

The question specifically mentions the multiprocessing module which sidesteps the GIL, so this part of your answer is invalid... – l4mpi Nov 20 '12 at 17:25
Maybe the GIL part is invalid, but the multiprocessing module relies on the OS scheduler, which will schedule in software and sequentually. You would need two processors that react to the same hardware interrupt which you aren't going to get without custom hardware. – James Thiele Nov 20 '12 at 19:02

Multiprocessing in Python: execute two functions at the exact same time

4 Answers4

Linked